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CHAPTER I 

A STATUS ASSESSMENT OF PERFORMANCE CONTRACTING 



Performance contracting has already established a rich semantic tradition. 
For some people it signifies "hucksterism," for others: a "hoax," and for 
still others "learning C.O.D." or "schools for profit." With equal ease, 
performance contracting implies free enterprise, a panacea, guaranteed- 
satisfaction-or-your-money-back, or a means to accountability. Attempts to 
demonstrate this technique have been dubbed "social experiments" or "quasi- 
experiments . " 

This history of performance contracting is both short and tempestuous. Many 
school districts entered into contracts of varying kinds, under differing 
conditions, which produced mixed results. Inadequate planning and prepara- 
tion coupled with overly ambitious expectations often doomed projects to 
failure. Other projects were successful in producing results which generated 
cautious enthusiasm and support. The status of all performance contracting 
in school districts and of the agencies which entered into those contracts 
should be of critical concern to educators throughout the country. Only 
after possession of this information can scnool district personnel make dis- 
passionate decisions concerning one of the most critical issues of our times. 

Performance Contracting Defined 

Basically, performance contracts differ little from "standard operating pro- 
cedure," whereby school boards purcha.se goods and services. In form, how- 
ever, these contracts have assumed a multitude of shapes. The contracted 
service in this instance is instruction. For this purpose school boards have 
engaged commercial educational companies or "^heir own teachers and administra- 
tors. The former are called external contractor::, the latter internal contractors. 



The contractor guai^antees to produce certain tanr;ible retsults, lie promisoci 
that students v;ill learn and claims he can demonstrat<-: this lerjrning "c-'^in" 
by comparlu/T scores on a test at the beginning and at the end of the program. 
In turn, the contractor can use the means he considered necessar^^, within 
certain constraints. 

The specifics of agreement between a contractor and the school board are 
drafted into legal form, hence the term ^^performance contract." The contract 
stipulates a minimum achievement level and measures payment according to the 
quality of job performed. To verify the results to the public, an outside 
evaluation or audit may be scheduled. If the accuracy of the results is 
attested, the contractor receives an amount of money scaled to the achieve- 
ment of the students. By incorporating bonuses and penalty sums, the con- " 
tracts depart from salaries traditionally tied to credentials and seniority. 

For a working definition of performance contracting, it is useful to consider 
the one employed by Joc^n Webster, Director of Contract Learning in Grand Rapids 
Public Schools (Michigan) : "A Performance contract is an agreement between 
a technical firm and the school system to produce specified results by a 
certain date Uoing acceptable methods for a set fee.""^ 

Turnkey, another concept that should be introduced, refers to operations 
started by an external contractor, but now carried on by the school with its 
own resources. The term is borrowed from the housing industry, where 
private- firms built public housing projects and then turned the keys back to 
the public for administration. 

Aims and Purposes 

Performance contracting is an approach, not a formulir.ed program. Ideally it 
is shaped by local conditions and therefore on the whole extremely diverse. 



Nonetheless, there are certain parts common to most performance contracts. 
George Hall and James Stucker, authors of Part I of the Rand Report on per- 
formance contracting, stressed this point: 

"While most performance contracting programs are remedial , they 
generally use an individualized approach to learning and focus on 
reading and mathematics , they are far from uniform. They are 
heterogeneous with respect to student populations, incentives, 
contractors' backgrounds and approaches, program organization and 
objectives. . . . Consequently, judgments of one program may not 
apply to others .""^ ( emphasis added) ' 

Diversity is apparent in the aims and purposes of the various performance 
contracts. They have sprung from a society in need of remedies for fla- 
grant human ailments. They testify to the tragic fact that masses of 
high school graduates are functionally illiterate. Drop-out rates are 
staggering in core poverty areas of major cities % As the Coleman Report - 
Equality of Educational Opportunity — indicated, disadvantaged students — 
impoverished and neglected — continue handicapped for life despite massive 
infusions of public monies into schools. 

By addressing these chronic problems, performance contracts have tried to 
prevent drop-outs, to improve basic skills such as reading and mathematics, 
to improve motivations and attitudes, to train for vocations, and to educat 
mentally retarded children. In short, the projects have sought to compen- 
sate for the often crippling effects of poverty, racism and neglect. 

The momentum for educational innovation is tied ^ot only to the conscience 
also to the pocketbook. With finances emerging as schools' major problem, 
administrators are shopping — "searching the marketplace," if you will. 
Their look to the outside reflects, in part-, the urgency of the situation 
and the inadequacy of the present system. 
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The federal government has provided the principal financial support for 
performance contracts. Most notably, the Office of Economic Opportunity 
conducted a nation-wide experiment in 1970-71, involving close to %7.Z 
million. Other funds have been allocated through the Elementary and Sec- 
ondary Education Act (ESEA - I963), Titles I, III and VIII, and from Model 
Cities (hud). State monies have been applied in Michigan, Colorado and 
now California. To a lesser degree, local resources have been used, 
especially nn Gary, Indiana, Dallas, Texas (foundation money)*, and Portland, 
Oregon. 

With money has come control. The broad range of fiscal sponsors has caused 
a variety of constraints for program operation (e.g. in Texarkana, Dallas 
and Grand Rapids). Hence, the .financial circumstances must be weighed in 
assessing the course of events. 

Performance Contracting in Development 

Developments in performance contracting have been shrouded in controversy, 
not from the very beginning, but from the end of the first year 1969-70^ 
At that time an audit of the instructional program declared that test re- 
sults were '^contaminated,'^ a result of the cciitractor's "teaching the test." 
The scene of this scandal was Texarkana, Arkansas. The issue remains clouded, 
since no agreement has been reached on the percentage of test items actually 
taught. 

The original goal of the Texarkana project v/as complex: to reduce the 
number of drop-outs, to assist in the desegregation process, and to improve 
reading and math skills. While labeled a "drop-out prevention program" 
(ESEA - VIII), Texarkana figured to measure its results indirectly, by 
test scores. Interim reports were glowing. (First thought to be represen- 
tative scores, they,, in fact, referred to the first students testing out of the 



- 5 - 



program.) ' As expectations soared, the stage was set for a great fall. 

The shock waves from the Texarkana tremor shook the educational circles o 
Efforts were made both to revive performance contracting and to check its 
growth. To many people's surprise, the Texarkana controversy failed to 
stop the launching of the comprehensive Office of Economic Opportunity (OEO) 
experiment. 

As the season passed, the results of Texarkana stood tall, although stripped 
of some laurels. Grade level gains in reading and mau. . r.atics seemed to be sub- 
stantial. Moreover, the drop-out rate was almost wiped out — only four 
students out of over 3OO dropped out. Other fruits strewn about in the 
storm were improved attitudes among students, parents and teachers. Yet, 
some 600 newspapers and 60 journals castigated the contractor (Dorse tt) for 
the newly-defined crime of ''teaching the test." 

Criticism of performance contracting had limited immediate effect. Along 
with the OEO experiment in twenty cities, over twenty other projects 
sprouted and shot up from the ground. In an assessment of one year of 
performance contracting,Reed Martin and Peter Briggs, then with Education 
Turnkey Systems, Inc., reported: ''Fifteen firms are performing under ^6 
contracts dealing with ^2p00 students at a total cost of 9 1/2 million 
dollars."*^ They qualify this estimate, however: "Compared to all this 

activity and the confident predictions of $150 million in projects, the 

k 

actuality seems almost meager." 

1969-70 The first year of performance contracting in education displayed 
some of its inherent possibilities. While the Texarkana project involved 
grades 7-12, the Portland, Oregon, contracts were concerned with inter- 
mediate grades 4-8. (Subsequently, performance contracting embraced all 
of grades K-12.) The subjects taught were reading and mathematics — both 



basic skills. The target student populations ran from 55 to 300 plus 
students, with maximum payments ranging from $500 to $135,000 (out of a 
to- mtract award of K27O,O0O), .At Texarkana an outside educational 
firm was engaged, while at Portland, native teachers, singly or ixi groups, 
were engaged, some subcontracting for materials and equipment, others 
relying on ^'traditional'^ classroom methods. The students selected were 
deficient in skills appropriate to their grade level; hence, the instruc- 
tion was remedial, 

1970-71 In its second year performance contracting mushroomed from an 
initial six to overy fifty projects. A major portion of this number com- 
prised the OEO program, an extensive experiment sparked by the Texarkana 
experience. This experiment concerned itself with the needs of a diverse 
student population r" black, white, Mexican-American, Indian, Puerto Rican 
Eskimo and oriental — all from poor and disadvantaged populations. The 
students were in grades 1-3 and 7-9; the subjects were again reading and 
mathematics, skills in which the vast majority were at le'ast two grade 
levels behind or below the national average. 

I 

In seeking geographic as well as ethnic representation, : the OEO project 
introduced two other factors — urban/rural differences and state/regional 
differences. Thirteen of the twenty projects were set in urban aur-eas — 
from Anchorage, Alaska, to Jacksonville, Florida; from Las Vegas, Nevada 
to Portland, "Maine. The dollar values of the eighteen external contracts 
ranged from ft2H,000 to S^^5,000 — considerably higher than the initial 
performance, contracts. (The projects- in Mesa, Arizona, and Stockton, 
California, v^ere technically separate from the others.) All told, the OEO 
effort involved 2^3,000 student participants (originally estimated to be 
28,000). 
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To insure a variety of approaches to' learning, six contractors were selected 
for the OhO <:ixperiment . These contractors were "based" in Georgia, New xMexicog 
Arkansas, Washington, D.C., and New York (2). 



When Texarkana stai^ted up, there were ten contractors bidding competitively 
against each other. One year later, at least forty companies were in the 
market, most of them small. (One report has it that one school received pro- 
posals from 89 companies when bids were solicited.) Walter Thomas, formerly 
of Combined Motivation Education Systems (CO-MES), classified the companies 
into three types: l) traditional school hardware manufacturers, 2) smaller, 
new corporations, and 3) traditional publishers. In the first tv/o years 
of performance' contracting (1969-71)1 there were at least nineteen contrac- 
tors actually involved in contracts. Outlined according to the above clas- 
sification, these firms were: 



**Westinghouse Learning 
Corp. 

Singer/Graf lex Corp. 

New Century (Meredith) 
*Thiokol Chemical Corp. 
*Hoffman Information 
Systems 



II 



Dorsett Educational 

Systems 
Learning Foundations * 
Quality Education Development 
*Plan Education Center 
Educational Solutions 
** Alpha Learning Systems (Alpha II ) 
^Combined Motivation Education 

Systems 
*Behayioral Research Labs 
Learning Research Associates 
Reading Foundation 
*Betti-Kit Corporation 
Learning Dynamics 



III 

*Educational 

Developmental Labs 
(McGraw-Hill) 
MacMillan Educational 
Services 
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Of these companies, about half (9) were operating under performance contracts 
in 1971-72-, two of them the result of company reorganization. (They are 
marked by single* and double ** respectively) « The names of the new companies 
(group 2 above) gives an idea of the thrust of the movement — a systematic 
approach to learning that takes into account behavioral psychology. 



Through the years I969-72 a total of 29 states '.rxperiiTienteci with this new 

technique. The largest number of contracts were negotiated in Virginia, 

Texas, Michigan, California and Oregon. In the Upper Midwest, only Ohio 

and Iowa were active in this area, each with one project. Aside from the 

OEO experiment described previously, the contract values varied from 

$500 to $6^-0,000. With rare exception, most notably Gary, the contract 

was in effect for one year. The subjects contracted for instruction included 

reading; reading and math; all subjects (Jacksonville and Gary); occupational 

skills and motivation (Dallas and Shreveport); and reading, mathematics and drop 

prevention (Dallas). Student populations in the projects ranged in size 

from about 60 (Portland, Oregon, and Keokuk, Iowa) to l4,26l (Philadelphia) 

students. 

A Catalog of Performance Contracts 

What would seem to be a relatively easy task — to identify all performance 
contracts to date — is, to the contrary, exceedingly difficult. Frankly, 
the artifacts have been stored in many museums, some to rust away in obscurity. 
The written remains, a small but con^sequential portion, are filed away in 
schools or else reproduced indiscriminately. In addition, the contents have 
both public and proprietary value (for companies, institutions and individuals). 
As a result, a search for complete data must take many directions. 

i ■ * • 

This report is based upon information supplied by schools, educational firms, 

evaluators, auditors, publishers, manag^^ment , support groups, state depart- 
ments of education, GEO and United States Office of Education (USOE). First, 
it might be useful to relate the parameters of this search. Schools are not 
equipped or })repared to respond to the many and varied requests for informa- 
tion. Second, contractors promote those reports that show the greatest success 
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■ • I 
or promise the most business prospects. Publishers' magnify those develop- 
ments of .greatest interest to the general public. State departments of 
education only occasionally receive reports on local expenditures of federal , 
monies, (it might also be added that State Educational Agency (SEA) infor- 
mation' provided on a questionnaire prepared in this study was generally 
incomplete, sometimes inaccurate, yet always useful.) Finally, USOE, for 
obvious logistical reasons, samples program reports and is unable to serve 
as a complete clearinghouse. 

In addition to\ the above-mentioned resources, certain individuals have been 
on the "cutting edge," and are thereby able to identify the course of 
the performance contracting movement. By piecing together these eclectic 
results, it is possible to list those cities in which performance contracts 
were negotiated. (Reference to city rather^than to school facilitates 
further investigation by interested persons). In keeping with an emphasis ^ 
on outputs, no attempt has been made to spell out the make-up of these con- 
tracts. Education Turnkey News has already done that job well. Moreover, 
they differ so vastly, sometimes so minutely from one another that a classi- 
fication would be counterproductive. Any broad scale comparison is precluded 

^\ 

for lack of a steiidard language, since contracts and evaluations are con- 
structed according to local customs, materials and tastes. S 

From another point of view, it would not be judicious to highlight or 

1 

to describe every project undertaken. They simply are not equivalent. In- 
stead, this report tries to call attention to the individual projects, inviting" 
closer examination by all educators. The examples cited are illustrative 
only, limited by the experience of the v/:|^iter. In the follov/ing listj 
those schools are starred if they were part of the OEO experiment. The cities 
in Virginia were part of a state-wide Title I project, i 

ERLC 

I 



- 10 - 



Sites of Educational Perfortnance Contracting 



*knchorage, Alaska 
Appalachian Virginia 
Athens, Georgia (Clarks Co.) 

Benton Harbor^ Michigan 
Big Stone Oap^ Virginia 
Bristol, Virginia 
* Bronx, New York 

Chase City, Virginia 

Clintwood, Virginia 

Coeburn, Virginia 

Colcord, Oklahoma (Delaware Co.) 

Compton, California 

Corbin, Kentucky 

Council, Virginia 



1970-71 
1970-71 

1970- 71 • 

1971- 72 

1970- 71 

1971- 72 
1970-71 

1970-71 
1970-71 
1970-71 
1970-71 
1970-71 
1970-71 
1970-71 



*+Dallas, Texas 
Denver, Colorado 
Detroit, Michigan 
Dothan, Alabama 

El Cajon, California 
+Englewood, Colorado 

Farmville, Virginia 

Fenton, Michigan 
+Flint, Michigan 
. Fontana, California 

* Fresno, California 

Gary, Indiana . 
Gilroy, California 
*+Grand Rapids, Michigan 
Greenville, South Carolina 
Grundy, Virginia 

* Hammond, Indiana 

* Hartford, Connecticut 
Hurley, Virginia r 

*+ Jacksonville, Florida (Duval Co.) 

Kalamazoo, Michigan . 
Kenbridge , Virginia 
Keokuk, Iowa 

La Junta, Colorado 
Lansing, Michigan 

* Las Vegas, Nevada (Clark Co.) 
Liberty-Eylau, Texas (Texarkana) 



1970-72 

1970- 72 

1971- 72 
1971-72 

1970-71 
1970-72 

1970-71 
1970-71 

1970- 72 

1971- 72 
1970-71 

1970-7^ 

1970-71 

1970- 72 

1971- 72 
1970-72 

1970-72 
1970-71 
1970-71 

1970-72 

1970-71 

1970- 71 
1970 s}xm 

1971- 72 
1971-72 
1970-71 
1969-72 
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* KcComb, Mississippi 1970-71 
Mesa, Arizona 1970-72 

+Hiami, Florida (Dade Co.) 1970-72 

Minco, Oklahoma (Delaware Co.) 1971-72 

Monroe, Michigan 1970-71 

Mount Clemens, Michigan 1970-71 

+Muskegon^ Heights,. Michigan 1971-72 

PJor a, Virginia 1970-71 

Norfolk, Virginia 1970-71 

Oakland, California 1970-71 

Ogden, Utah; 1970-71 

++Philadelphia, Pennsylvania 1970-72 
Pine Ridge, South Dakota (Shannon Cop) 1971-72 

Portland, Maine 1970-71 

+Portland, Oregon 1970-71 

Providence, Rhode Island 1970-71 

i 

* Rockland, Maine 1970-71 
Roxbury, Massachusettes (Boston) 1970-71 

San Diego, California 1970-72 

St. Joseph, Louisiana 1970-71 

Savannah, Georgia 1970-71 

+Scottsdale, Arizona 1971-72. 

* Seattle, \\lashington • 1970-71 

* Selmer, Tennessee (McNairy Co,) 1970-71 
Shreveport, Louisiana 1971-72 
South Hill, Virginia 1970-71 
Stockton, California 1970-71 



* Taft, Texas 1970-72 
Texarkana, Arkansas 1969-72 

Vansant, Virginia 1970-71 

Victoria, Virginia 1970-71 

Wayne, Michigan 1970-71 

* Wichita, Kansas 1970-71 
Wise, Virginia 1970-72 



* - part of OEO experiment in performance contracting 
+ - more than one project 

In sum, there are 75 cities in Jl states represented 
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1971-72 There were external performance contracts in operation at the 
following locations: 

Benton Harbor, Michigan i 

Dallas, Texas - 2 

Detroit, Michigan 

Dothan, Alabama - 

Flint, Michigan . ]_ 

Gary, Indiana - 

Grand Rapids, Michigan- -------^-3 



Jacksonville, Florida (D.uval Co,) 1 

Lansing, Michigan - -- -^---.----]_ 

Miami, Florida (Dade Co.) -2 

Minco, Oklahoma 1 

Muskegon Heights, Michigan 2 

Philadelphia, Pennsylvania ™ i 

Scottsdale, Arizona - -- - 3 

Shreveport, Louisiana 

22 

Internal contracts were in effect in: 

Bristol, Virginia -1 

Dallas, Texas - 

Denver, Colorado- -]_' 

Englewood, Colorado . 2 

La Junta, Colorado- -----------1 

Mesa, Arizona - 

Miami, Florida (Dade Co.) ' 2 

Pine Ridge, South Dakota (Shannon Co.) i 

22 

Turnkey operations were in force at: 
Flint, Michigan - 

Grand Rapids, Michigan -2. 

Grundy, Virginia- - -- -.-----^-_]_ 
Hammond, Indiana- - 

Philadelphia, Pennsylvania : 1 

San Diego, California -1 

Taft, Texas 1 

Texarkana/Liberty-Eylau 1 

Wise, Virginia- 

To 

Other contracts as yet unclassified include: 

Fontana, California - - - -1 
Greenville, South' Carolina 1 
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All told, there were at least 5^ performance contracts in effect (plus 10 
Turnkey operations). There are also unconfirmed reports of contracts in 
Chicago and Muskegon, Michigan, Current operations are sca^ tex 3d throughout* 
13 states. 

Although the direction of performance contracts has been in the area of 
reading and mathematics, there are indications that their use may be expanded 
for purposes of improving motivation and attitudes, teaching mentally re- 
tarded children, e,g. in Grand Rapids, and especially providing vocational 
training* For example, Shreveport, Louisiana, is following in Dallas' 
footsteps in contracting for vocational training for both boys and girls, 
Dallas, meanwhile, has spawned another program, this time a career develop- 
ment program — the Skyline project. These varieties that have evolved 
bear out the observation made by Stucker and Hall: "The real strength of 
performance contracting is its versatility — its potentiality for use in 
a wide variety of contexts,"^ 

Information Sources 

Lacking a single, definite work on performance contracting, it is appro- 
priate to elaborate on the available sources of information. First, it 
should be admitted that the whole story will probably never surface. The 
experiences were too separate, too short-lived, too ill- and unreported, 
too controversial, too small and too momentary in relation to other social 
and economic conditions crying for attention. Moreover, many of the matters 
are considered personal, private and proprietary as well as public. Under' 
the threat of lawsuits — the suspended sword of Damocles — no party 
will divulge complete information, at least "not for publication," or as 
one authority put it: "In damn few private conversations," This makes 

ERIC 



published reports terribly incomplete, hinting at best at the actual situa- 
tion. Second, because of the business interest involved, the material 
may be classified, as noted by Martin and Briggs: "Some of the problems 

encountered are getting actual figures, as opposed to whau the firm wants 

7 • 

its costs to appear to be." 

A basic source of information on performance contracting is the year-end 
report prepared at the local level. The standards for these reports, 
however, vary tremendously. Often they are distillates of limited value 
except as written record or justification required by a school administra- 
tor or government agency.' Prepared usually at a "safe" distance from the 
project, they display selective retention in choosing to report some things 
and to ignore others* Nonetheless, as the most readily available source, 
apart from personal communication^ these reports stand as monuments to 
the past. 

Another source is the large number of articles appearing in sporadic 
fashion in a wide range of journals, educational and otherwise. With rare 
exception, these have been two-dimensional snapshots. Their purpose seems 
to be to influence public opinion or to demonstrate professional awareness — 
being on top of current educational developments. LocaJL newspaper accounts, 
while timely and detailed, have unfortunately not circulated widely. As 
a result, there are gaping holes in the chronicle of performance contracting. 

A third recent source is the product of major research efforts by Rand Cor- 
poration and the Office of Economic Opportunity. Rand analysts spent l6 
months exploring the concept, investigating the field and developing a guide 
for decision-makers in education. OEO, together with its principal analysis 
contractor Battelle Memorial Institute, has completed its 12+ month study 
of the OEO experiment. These publications, voluminous and of varying statistical 
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sophistication, provide the greatest supply of material to date -oh which 
t'o base an examination of performance contracts. 

A fourth source is the increasing volume of dissertations and books, largely 
the product of personal and corporate experience. They augment the earlier 
work of National School Boards Association (NSBA), Lessinger and Education 
Turnkey Systems, Inc. This activity was referred to, perhaps facetiously, 
as a "knowledge explosion", since at the end of year I, "Three books were 

8 

published, ranging in price from tt3 to S'95» and this newsletter was begun." 

The chronicle of performance contracting still has to be written. The 
scribes are many, the scrolls — originals and copies — as well. Foremost 
among the documents is Education Turnkey News , which for various reasons 
(altruistic and commerical) began to report on events and development. This 
publication suffered from no lack of scope, claiming a capability to survey 
the field, to run a classified advertising section, to analyze the news, 
to cover special subjects in depth, and to aid educational planning* 

The newsletter began in April, 1970, and ran through March, 1971. It 
reported basically matters relating to performance contracting, but included 
topics like "educational engineering" and education vouchers. The final 
three issues developed a useful guideline for planning a contract and a 
summary of results in year I, as interpreted by Briggs and Martin. . Especially 
the last issue is revealing in its disappointedl,, perhaps disillusioned tone. 

As a documentation of events. Education Turnkey News is without peer. It 
was published by Education Turnkey Systems, Inc. (ETS), a management support 
firm active in analysis, evaluation and planning. Geared partly to company 
operations, the newsletter solicited innovative efforts from schools, and 
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■introduced new learning companies — theii- personnel a..i activities. In 
addition, it highlighted major conferences, announced contract awards on 
the Capitol scene and provided a forcecast of events. The eff^^ct of this 
work was a sustained, reasonably complete account of the action* 

Education Daily also tracked the course of performance contracting. It 
fills in a lot of holes by documenting USOE and OEO actions, including 
contract awards, personnel turn-over, press conferences and news releases. 
By reporting from Washington, D.C., this publication tied performance con- 
tracting to the congressional scene. 

Major magazines are the source of greatest disappointment. Their articles 
by writers of wide-ranging expertise addressed only peripherally the matter 
of performance contracting • Bumstead, Elam, Mecklenburger, Wilson and 
Willingham are clear exceptions. Moreover, the colizmns by Mecklenburger 
in Educational Technology and by Blaschke in Nation's Schools took over the 
function of the ''ETS^' newsletter and reported development on a regular basis. 
Aside from these items, the "dirigible** of performance contracting (see 
Mecklenburger article in appendix) was tied down by a string to most pub- 
lished articles. And the connection was remote. 

Elements of Controversy ^ ^ 

On the whole, performance contracting has not held the public attention 
for long, except for Texarkana, OEO and Gary. It has had the flair of a 
bandwagon approachj which has fostered wholesale adoption as well as. the 
taking up of positions opposing the movement. As a result, controversy 

has become the trademark of performance contracting. 



The fires of controversy have been fanned by "zealots" (ad-and ad-ment alike), 
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"climbers," limelighters, vindictive parents, defensive teachers, administra- 
tors of expediency, insensitive technicians, and irresponsible public rela- 
tions officers. Some of the basis for controversy can be summarized as 
follows: 

1, Performance contracting involves extensive use of para- 
professionals — uncertificated people. But as recent 
evidence seems to indicate, this practice has a sound 
theoretical basis. It just may not be the case, as is 
commonly assumed,, that the teacher-student ratio is the 
critical factor in learning. Instead, how the avail- 
able personnel are deployed may be most imp.ortant. 
Teachers, especially through their organizations, have 
felt their positions threatened by increased use of 
paraprofessionals, 

2, Performance contracting emphasized individualized in- 
struction , made possible by varying combinations of 
equipment and materials — usually "programmed texts" 
but also sometimes "teaching machines," It has been ■ 

a long-standing tradition of schools to delay the intro- 
duction of technological progress into the classroom, 

3» Performance contracting regards motivation as the key to 
learning. Thus, it encourages treating kids and adults 
alike, by rewarding them with tangibles, such as green 
stamps*, play money, candy, transitor radios, or with 
intangibles, such' as free time, recreational privileges. 
This practice shocks traditional purists who believe 
only in self-motivation, while ignoring the effects of 
complex . social and economic conditions. Purists pre- 
fer the more subtle and accepted incentives of grades, 
extra-curriculars, degrees, a good job, high living 
standards and status, 

k» Performance contracting pays according to end results ; 
consequently, contractors run high^risks, unsure of 
making a profit, much less meeting expenses. On the 
other hand, contracts offer the certainty of knowing 
what you paid for. You come away with a receipt in 
your hands. Teachers who relish secure and non-com- 
petitive conditions take issue with this flexible 
approach, 

5, Finally, performance contracting involes a new way of 
thinking about schools, complete . with new roles, dif- 
ferent environments and a jargon of its own — learning 
managers, curriculum managers, rapid lesirning centers, 
contingency management. This novelty of form upsets the 
status quo and its dependents. 
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CHAPTER II 



REVIEW AND ANALYSIS OF SUBJECTIVE REPORTS 

The temptation is strong to label performance contracts as either 
"successes" or "failures." To do so, however, is to over-simplify 
matters. Hard data are available in the form of test scores, head counts, 
rates of attendance and drcp-out, and incidents of vandalism. But sub- 
jective and affective considerations should be regarded as well. News 
releases have generalized extensively, branding projects as: a "flop," 
"mixed success," "success," "encouraging," or "disappointing." Similarly 
performance contracts have been summeirily debunked for their lack of 
"dramatic" or "striking" results. 

It is important to remember at this point the heterogeneous character 
ascribed to performance contracts. There are circumstances peculiar to 
each one. Schools and communities differ in the degree to which they 
have recognized and assessed their needs. They differ ^n the expectations 
they hold for schools. They differ in resources, finances, personnel and 
students. And they differ in commitment. These differences are ultimately 
reflected in the types of contracts negotiated as well as in the outcome 
of those agreements. 

Viewpoints of Selected Writers 

The observer, probably closest to the scene of performance contracts in 
actual operation, has been James Mecklenburger, research assistant for 
Phi Delta Kappan . Together with John A. Wilson, a fellow graduate student 
at Indiana University and now with the Indiana SEA (Department of Public 
Instruction), he has written objective, yet personal accounts based on 
first-hand experience in Gary, Cherry Creek (Englewood-, Colorado), and 
»Grand Rapids. In their September, 1971« article for Saturday Review 
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entitled ^^Learning C.O.DJ^ they related some effects of performance con- 
tracting: 

"Some contracts have shattered complacency, inspired creativity, 
improved learning, and turned the spotlight of public attention 
on the quality of classroom instruction. But others have in- 
spired greed and chicanery, created poor environments for child- 
ren, and fomented unhealthy dissension'' .9 

From their experienced point of view, they recommend looking at contracts 

in the following way: 

"Some projects undoubtedly will appear to do very well, others 
will be instructive as failures. All should be regarded with 
the kind of hopeful skepticism that greets the first trial of 
any new invention. "10 

Robert Stake and James Wardop of the University of Illinois (Center for 

Institutional Research and Curriculum Evaluation — CIRCE) have echoed 

the sentiments of the Rand Report. They have sharply criticized the 

testing procedures. Stake put it this way: 

"At first performance contracting seemed almost a haven for 
the misinterpretation of scores- Contracts have ignored 
1) the practice effect of pre-testing, 2) the origins of grade 
equivalents, 3) the "learning calendar," 4) the unreliability 
of gain scores, and 5) regression effects. "11 

In particular. Stake and Wardrop contested the instruments used in per- 
formance contracting to demonstrate "growth." The problem is acute, 
because test scores are the basis for payment to the contractor. Stake 
pointed out that probable errors of up to one year can result from 

measuring short-terra individual gain by means of a standardized achieve- 
12 

ment test. Criteria -referenced tests, while more appropriate to particu- 
lar learning objectives, have not been sufficiently developed or field tested. 

The dilema of testing in performance contracts is summed up essentially in 
the following: 

"It is often unrealistic to expect a project director to either 
find or create paper and pencil test items, administrable in an 
hour to large numbers of students, by persons untrained in 
psychometric observation and standardized diagnostics, objectively 
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scorable, valid for purposes of the performance contract, and 
readily interpretable."13 

Despite their reservations about the evaluation of the performance con- 
tracting programs, Stake refrains from a wholesale condemnation of the 
method: 

"Without yielding to the temptation to undercut new efforts to 
provide instruction, educators should continue to be apprehen- 
sive about evaluating teaching on the basis of performance 
testing alone. 

From another standpoint, this time that of Charles Blaschke, president of 
"ETS" and the propelling force behind the theory and process of performance 
contracting, the year 1970-71 might be described in these terms: "Heartening 
and disheartening; encouraging and discouraging. More importantly, perhaps 
is that performance contracting and turnkey operations actually exist. 
Texarkana is history. ""^^ 



At this mention of Texarkana, it is worth considering the words of 

Robert E. Kraner, president oi iCpic Diversified Systems, Corp., the auditor 

for Texarkana (1969-70). In the fall of 1971 he wrote: 

"The Texarkana experience strengthened the total concept of 
accountability through performance contracting. The evalua- 
tion and audit functions of the contrfxt were allowed to demon- 
strate their value in a demanding situation. Not only did the 
evaluation and audit functions serve as a formal check to con- 
tractual obligations, but they provided basic information for 
project development and operation during this second year. 

Today, the Texarkana Project is operating as one of the out- 
standing ESEA, Title VIII projects in the nation. With the 
same safeguards built into the project, evaluation and educa- 
tional audit, the public should feel a higher level of as- 
surance that monies are being expended properly and effectively 
by its manage ment."^^ 



It should be mentioned in passing that Kraner 's position is not universaJly 
shared, partly because of the abrupt shift in instructional programs, and 
partly because of the high start-up costs of the follow-up program. 
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In terms of cost and the cost-effectiveness of performance contracting, the 
final OEO report on its 1970-71 experiment will have significance. The 
management group for that project was *'ETS", whose president Blaschke re- 
leased preliminary results from a cost analysis of selected programs. His 
figures indicated that the average rates of achievement in mathematics and reading 

for underachieving students were doubled at a cost slightly more than 

17 

existing costs per student year per subject. This meant, in effect, that 
the projects were financially feasible, relative to the current costs 
of remedial programs in ^hese areas. The report, however, did not determine 
the actual effectiveness of the projects, since the aGhievement data were 
not available for analysis at that time. 

The attitude of OEO during its experiment, discussed later in greater detail, 

was ambiguous. For example, John 0. Wilson, Assistant Director for Planning, 

Research and Evaluation, stated his position in mid .course: 

"We are telling those superintendents flatly that we would not 
advise it (performance contracting). We really know very 
little about this concept, far too little to indicate even 
optimism. I am apauled when I read that schools already are 
spending millions of dollars for this entirely unknown quantity." 

Strangely enough, this OEO office was in the midst of its $6.5 million 
experiment in 20 cities, when this opinion was expressed. 

Opinion Polls and Surveys 

Another index to the effects of performance contracting is the result of 

scattered polls and surveys. In May, 1970, a poll of school superintendents 

in the 50 states indicated that kl% thought that performance contracts with 

19 

private firms as in Texarkana should be encouraged. 

The next poll in November, 1970, the by now familiar NSBA (National School 
Board Association"^ poll of school board members in ^9 states was more supportive 
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of performance contracts. Of the board members polled, on-^— third strongly 
favored the con'j^^pt of pei'forrnance contra':tinf^; one-th.ird favored the idea, but 
with soM'.^ 3'"::''i v.'^». LoJir; wtiii-j orif-Ltrird o|i{;ou"'i tha notion. 

The Tnird Annual Gallup Survey of Public Attitudes Toward Public Schools in 

'mid~1971 pointed out that kk% of the public favored the idea of performance 

Contracting, while 23% opposed it. However, a substantial portion — 23% — 

21 

expressed no opinion. Thus, the general public tended to support the 
school board members' position. 

A teacher opinion poll conducted last fall and reported in December, 1971 i 

in Todays Education showed that: 

7.5% strongly favored performance contracting 

30.5% tended to favor performance contracting 

25.7?^ tended to oppose performance contracting 

22.0% strongly opposed performance contracting 

1^.^% were undecided 



In summary, 38% of the teachers found the idea favorable, while ko% were 
22 

opposed to it. (The results of this poll may not be too reliable, since 

no sample data were given.) This report concluded: . .public school 

teachers in general do not believe that the type of competition for money 

23 

customary in the business world should be applied in education." 

Hence, the surveys considered implied an approval by the general public and 
school boards, alongside mild disapproval by superintendents and teachers. 
(This sentiment is shared by a company president who recently said: "Nobody 
seems to like it but the customers that buy it.") 



Reaction of Professional Organizations 



Another indicator of the credible success of performance contracts has been 
the reaction of major organizations. For example, the American Association of 



School Adm inistrat ors (AASA) , while affirming the principle of accounta- 
bility, cautioned: ^ 

"When school districts contract with commer cial organiz ations for 
part or all of the education program, the result obtained may appear 
to be the desired one, although it is all too likely to be specious." 

Consequently, the AASA endorsed experimentation in contracted learning "only 

under strict supervision by persons who' know in depth , . . (emphasis added) 

The American Federation of Teachers (AFT) has been particularly vocal in its 
criticism of performance contracting, expecially in Gary, Providence, 
Philadelphia, Boston and Denver. The AFT opposes the usurpation of educa- 
tional policy by private industrial entrepreneurs; fears the possible 
monopoly of big business*, condemns dehumanizing procedures and "teaching 
the test", practices; is resistant to subversion of the collective-bargaining 

process; opposes a reduction of teacher input; and is concerned for the 

25 

competition that may be shown among school personnel. 

The AFT reacted strongly to performance contracting. Its position was 
illustrated by cartoons in the American Teacher and the AFT Non-Colorinpi: 
Book on performance contracting. At the same time, by opposing the concept 
and OEO»s involvement in educational affairs, the AFT precipitated out issues 
that were unresolved and unattended for a long time. There was, however, con- 
troversy resulting from misunderstanding and the lack of communication 
which produced a damaging effect. As an OEO official acknowledged, the AFT 
reaction was expected because of the thorns in performance contracts - the 
effect on collective bargaining and on employment practices. 

i 

The Council of Chief State School Officers (CCSSO) has expressed its concern 
for the legal implications: 
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^^This involvement of the private sector in the conduct of educa- 
tional programs should in no way lead to the abdication ql state 
and local responsibility for management and accountability for 
public education in the statr?."26 

Probably, one of the least emotion-ridden and best formulated po;.iitions is statin) 
by Rev, C. Albert Koob, President of the National Catholic Educational Associa- 
tion: 

'*ln their present forms and under present circumstances, neither 
the Educational Voucher Plan nor Educational Performance Contracting 
can be considered either singly or as a combination, the best or 
only solution to current educational ills. But, because they repre- 
sent the studied efforts of highly regarded individuals in education, 
government, and industry, it obviously is in the national ' interest 
to provide every opportunity to determine the impact- for good or for 
bad- that these devices generate, with a view toward subsequent ex- 
pansion, alteration, or abandonment, as their tested merits indicate. 
It is understood, of course, that prudent and proper supervision and 
safeguards are to be exercised in all such experimentation and in all 
such innovative programs ,"27 

The National Education Association (NEA) softened with time. Early, the NEA 

listed nine (later eight) detailed conditions as prerequisites for its 

approval of such contracts. Teachers must be integrally involved at all 

stages; a variety of tests must be used; the community and other professionals 

must help formulate learning objectives; there must be turnkey provisions - 

i.e. the school must ultimately take over the program in all aspects; school 

personnel must be used as much as possible; all personnel must 'be properly 

certificated; (programs must be truly innovative); existing contracts must 

28 

not be violated; and copyright laws must be observed. As is apparent, 
the concerns of NEA were real. They did not, strictly spealcing, relate to 
the GEO involvement in experimentation. 

To NEA Presidents > Mrs. Helen Bain and later Don Morrison, performance con- 
tracting resembled other simple, cheap solutions to complex and costly 
problems." In their view, no prograni can replace a good teacher working 
with small classes and with adequate time and materials. Another NEA spokesman, 
lobbyiPu John Lumley, assailed the GEO project as "a poor use of GEG funds." 
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iie supf.ortod only the idea of local level performance contracts, whore "... 
school people and the schools are involved, not private, profit-making firms." 

The National Association for the Advancement of Colored People (NAACP) has 
passed a resolution in favor of performance contracting. It should be men- 
tioned that projects in Texarkana and elsewhere have been instrumental 
in desegragation efforts. 

In general, it has been highly popular for major organizations to take a 
stance on performance contracting. The extreme forms are represented in 
the New York state regulation prohibiting the use of state or local monies 
for performance contracting and in Michigan's accountability efforts, 
making S500,000 of state Section III monies available for performance con- 
tracts. Frequently, "positions have been ill-founded, based on prejudgement 
and insufficient or faulty information. The Educational Policies Commission 
of the Minnesota Education Association has adopted a position that describes 
the profitable nature of performance contracts, that questions the commitment 
of business to children, and that supported the expertise of schools and 
their personnel. The position conclude;^: 

"It is doubtful whether any group of industrialists or educators 
will develop a new method which will replace a good teacher who 
has small classes, adequate preparation time and good supervision. 
Providing the student with these advantages is costly, and it always 
will be. "31 

Newspaper Capsule Reports 

The following capsule reports on performance contracts illustrate the sub- 
stance of most news releases thus far. The Dallas projects were particularly 
successful. Thiokol Chemical Corporation, for example, undertook a vocational 
training program for 150, students (coed), consisting of auto mechanics, 
drafting for girls, and machine shop training. Thidcol guaranteed that the 
students v;ould achieve entrance level performance as apprentices, assistants 
"or helpers, by Department of Labor standards. As a result of the contract 



- 26 - 



65^ em3rged at the desired level, another 30?^ completed on-the- job-training, 
while only 5% failed. Another Dallas contract for motivational improvement 
succeed-d by raising the attendance rate from 72?o to 8?^, by improving 
attitude=s, and by reducing the drop-out rate from 8% to 55^- 

The Gary pro.iect resulted in 73% of second- through sixth graders scoring at or 
above the national norrr.s in reading or matheihatics or both - a complete turn- 
about from the previous year- (A clearer reporting of the prograrr. would have 
shown that 55% in reading or mathematics, 52% in mathematics only, and G% 
in reading only met standards of national norms.) .Firsi-p:rader^5 acheived an 
average of ]--7 year's growth, with 1.5 ye^^rs an avorage for the other s:rades- 
(Measurement of Ei^ains in the first grade has been questioned by B.laschke, 
Webster and Hall.) 

The Philadelphia project was the largest to date - l4,26l students. Marred 
by a teachers' strike and a delayed shipment of materials, the pi-'oject pro- 
duced a year's growth in reading for 3^% of the pupils, while 35% failed to 
reach that level. Significantly, 30% were discounted fi^om the payment schedule 
for not attenting 150 days. According to the head of Behavioral Research 
Laboratories (BRL), Allen Calvin, the results were still remarkable: "These 
are the best results anybody's gotten with inner-city kids. Anyv/here!" 

Although the Portland, Oregon, contracts ran so well that they were continued 

in the summer, director James Holmes disuaded their wholesale adoption: 

"Few teachers are willing to ran the risk of a true tough performance contract." 

Texarkana in I969-7O showed 250 out of 550 students improving 1.5 grade 
levels in reading, along with a reduction of drop-out rates from 20% to 2% 
(Dorsett's figures). In the next year, EDL (Educatior.al Development Labora- 
tories of McGraw Hill) lowered the drop-out rate from 8% to 5%j met the minimal 
achievement level in criterion-referenced tests, but produced only a 2h% success 
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rate with standardized tests as the measure. 

The well planned Virginia state-wide project produced "disappointing" results, 
despite the claims to the contrary by several of the participants. The stu- 
dents showed no advantage in reading gain over their peers in regular 
classrooms, despite the painstaking efforts made to develop the program. 

Overall, the actual costs per grade level gain were less than normal, largely 
because of the ambitious goals and high minimum achievement levels. In 
Blaschke's cost analysis of performance contracts, he pointed out the marked 
contrasts in the amount of money expended in experimental programs versus 
traditional programs for teacher pay (55% vs 75%) and for books and audio- 
visual materials (17?^ vs 2^). On the basis of these data, Blaschke con- 
tended: "If schools turnkeyed (adopted instructional equipment and instruc- 
tional prograTis), operating costs would be less than present costs/student/ 
subject in about one-third of the cases and somewhat greater in the rest."-^^ 

Robert W, Locke, late Executive Vice-President for McGraw-Hill Book Company, 
allayed some of the financial worries over performance contracting when he. 
said: 

"Nevertheless, I wouldn't get too uptight about the prospect of paying 
more money, because the theory of performance contracting is that 
educational results will be better. Remember the only way a performance 
contract can cost less is to fail. Conversely, it will be relatively 
economical if it succeeds. "33 



CHAPTER 111 



REVIEW AND ANALYSIS OF RAND h]VALUATION REPORTS 

The Rand Corporation study of performance contracting in education was com- 
pleted in March, 1972. Consisting; of three parts totaling ten volumes 
(2-6-2), it represented the results of the investigations of James Stucker, 
George Hall, Polly Carpenter, Arnold Chalfant, M. Rapp, G«C. Sumner, and 
S.A. Haggart. The study was made pursuant to a contract with the Department 
of Health, Education and Welfare (HEW). The sole-source contract was announced 
in July, 1970 and was scheduled to run for l6 months at a cost of $300,000, 

Part I - Concepi; and Theory 

Dated May, 1971» Part I by Stucker and Kail examines the concept of per- 
formance contracting and then attempts to derive the theory. It is partially 
based on similar experiences in the Department of Defense (DOD), National 
Aeronautics and Space Administration (NASA), Job Corps and hospitals. 

In discharging its responsibilities, a school board has to decide what it can 
do on its own and what it needs to purchase. Contracts are instruments. 
They can be used to purchase goods an.d services, or as the authors put it 
"resources and results," 

Stucker and Hall describe four kinds of contracts: fixed fee contracts 
for resources, fixed fee contracts for results, scaled (performance) contracts 
for resources, and scaled (performance) contracts for results. Each of the 
four is suited for particular tasks. 

There are two basic criteria against which to weigh contracts. Of central 
importance is the matter of authority, which is delegated by the board. In 
general, the board surrenders little decision-making authority in fixed fee 
contracts. Scaled (performance) contracts, however, imply greater flexibility 
for the contractor and hence greater authority. 
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The author suggests that a performance contract permits multiple outcomes, 
instead oi a single outcome. Anticipating certain adjustments then, this 
contract scales the price to fit the end product. In this way, the payment 
corresponds to the level of perform,ance. Fixed contracts, on the other 
hand, are 3f3ss flexible, specifying a solitary outcome and a flat fee. 

Part I also calls attention to certain problems encountered in performance 
contracting, in particular the matter of testing and measurement. It sounds 
a prophetic warning: "Designing and implementing such evaluation, may turn 
out to be the most difficult problem the performance contracting movement 
must face, . 

Stucker makes an interesting attempt to derive the theory of performance con- 
tracting by integrating the theory of contracts (sales and employment) with 
the theory of incentives. The matter should be pursued in greater depth. 

Part II - Case Studies 

The results of the field investigations of Rand's Team are found in the 
second part (6 volumes) called Case Studies in Educational Performance Con- 
tracting . The sites under study were Norfolk, Virginia, Texarkana, Arkansas; 
Gary, Indiana, Gilroy, California, and Grand Rapids, Michigan, Dated December, 
1971, the case studies report on performance contracts in the school year 
1970-71. 

Conclusions and Implications 

Volume I of Part II is entitled Conclusions and Implications ^ drawn together 
by Carpenter and ^Hall from the other five volumes. It provides the basis 
for the following summary. 
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The instructional programs were not "off the shelf" stock; they were still 
in the process of development. The basic and common goal of this development 
v/as individualized instruction. In this way, performance contracts came 
to serve as agents of change in general, despite their preoccupation with 
skills. Contrary to accusations, the program produced little evidence of 
dehumanization. 

The cognitive achievements of these contracts were "respectable," but not 
"dramatic". While discussing the formidable problems in determining test 
scores and gains, the authors stress the need for considerable work in the 
area of criterion-referenced measures. Progress in this area will aid 
decision-makers, who ultimately mast choose the instruments by which to gauge 
learning. 

The contracts entailed a wide variation in costs, but generally totalled 
more than in "conventional instruction." However, Carpenter and Hall are 
quick to point out that the costs are roughly the same as for other ESEA 
Title I programs. The basis for this claim is the standard replication cost 
analysis set up by S.A. Haggart. 

The mark of success in performance contracts is student learning. Testing is 
utilized to make this determination whereas evaluation assumes the function 
of validating results. In general, evaluation schemes at the five sites 
were haphazard or non-existent, sometimes as a result of unavailable baseline 
data. 

As a research and development tool, performance contracting has shown promises. 
Its success, however, ultimately turns on the presence or absence of a re- 
spected and influential "sponsor," as documented in these cases. Early 
teacher involvement and a flexible contract also help to insure success. 



Along legal lines, disputes may arise with respect "to public control of the 
programs (or the lack thereof) and on the matter of merit pay. Finally, 
community relations, thus far unattended have to improve for the sake of 
the program. 

The author Is note that the LSC's (contractors) do not seem .to have generated 
large profits so far. (Brian Frider, president of Alpha II, referred to 
this situation as "the case of the missing pile of profits.'O Some companies 
have generated follow-up programs, usually, however, not on a performance 
contract basis. (Blaschke asserts that companies are ideally working them- 
selves out of business.) Contractors seem to prefer a consultant position, 
having found performance contracting a way to break into new markets and 
to get visibility. 

In summary, Carpenter and Hall cite the major advantages of performance con- 
tracting: it facilitates the introduction of radical change, it increases 
emphasis on accountability for learning among all the parties involved, and 
it has brought new LSC^s into the field. 

On the other hand, there are disadvantages: high costs and complex 
management, a narrovj focus on skill development, and an exacerbation of old 
problems like teacher status and test usage, 

Norfolk, Virginia 

Polly Carpenter prepared volume 2 of Part II on Norfolk, one of Virginia's 
state-wide Title I projects, which yielded "disappointing" results. She 
diagnosed a basic flaw in the .program design — namely, a mismatch between a 
curriculum emphasizing reading attack skills and a standardized test emphasizing 
reading comprehension. There were other difficulties as well — an "inadequate" 
evaluation plan and obstacles to collecting necessary data. By running a 
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comparison with a regular Title I remedial reading program, she calculated that 
performance contracting would reduce costs by 255^. Fiaally, as a result of 
the program, "radical'* departures from traditional practices were introduced 
into the school system. 

Texarkana, Oklahoma 

Rand, in the persons of Carpenter, Chalfant, and Hall, made the pilgrimage 
to Mecca — i.e. Texarkana. By sifting through the remains of I969-70, they 
found the relics of the original program. It involved these features: l) new 
cost-effective technology, 2) a performance contract, 3) a management s'.pport 
group, ^) competitive bidding by LSC*s and 5) independent evaluation and audit. 
They detected three problems in the first year design and operation: 1) an 
evaluation plan out of tune with the instructional program, 2) an unclear defi- 
nition of roles for the support groups, and 3) inadequate preparation for 
turnkey. 

As a result of the "teaching the tecV^ scandal, major changes were brought abouto 
More direct emphasis was put on preventing drop-outs ;. support roles were simpli- 
fied; and payment was connected not only to standardized test scores, but 
to reduction in the drop-out rate, to achievement on criterion-referenced 
tests and to cost-effectiveness. 

The accuracy of this report is attested by project director Martin Filogamo: 

'^We are of the opinion they (Rand) have done an excellent job reporting the facts.'^ 

Texarkana retains its importance as the model for performance contracts, as 

observed by Briggs and Martin: "It is saf e^ to say that since Texarkana, no 

significantly new approaches have been offered by the firm.s. Each project has 
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the same old wine in new bottles.""^ 
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Gary, Indiana 

Gary caught the eye and the affection of the Rand team. Hall and Rapp called 
it "the most innovative of the contracts.'* As the center of controversy — 
internal, local and national — Gary was challenged almost beyond its strength. 
It had spats with its LSC, with the local teachers' union and the AFT at large, 
with the Indiana Department of Public Instruction, and even with government 
officials. Yet despite these intitial confrontations, the authors concluded: 
"With some effort it appears that most of the legal, administrative and personnel 
difficulties can be resolved." 

As a multi' year contract, the Banneker project in Gary must be looked at 
over time — a "wait and see" approach — recognizing simultaneously the im- 
pressive gains in the first year (1.5 years). Hall and Rapp cite also the 
cost potential of the program, figuring it to exceed conventional costs by 
no more then 5?^. For these reasons they strongly recommend: "It deserves 
the attention of everyone interested in tho current educational scene." (These 
comments take on more meaning when considering the remark of Brian Fitch, 
project director at Bannecker, that Rand viewed the project iri the "incuba- 
tion stage." Since that time program development — "the right materials 
to the right kids at the right time" — has come a long way.) 

Gilroy, California 

M. Rapp developed the report on Gilroy, California, whose program was also 
a source of disappointment. She identified special problems in a curriculum 
for Spanish-American children, testing and evaluation difficulties, proprietary 
cost information and the need for a mid-course correction. The program, while 
not achieving its stated goal, did effect program change in the direction of 
individualized instruction. It also maintained the status quo, in that the 
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differential between the experimental group and tlie control group did not 
widen over the course of the instruction. 

Grand Kapids, Michigan 

The volume on Grand Rapids, written by G.C. Sumner, is steeped in local flavor 
In convincing style, the report identifies and credits the "sponsors" of the 
three performance contracts in effect. Necessarily, one of them, the GEO 
project, labored under the "clamp of silence," as Joan Webster put it, and 
behind the ''secrecy wall," as 3r±aa Frieder put it. There is, however, still 
some basis for comparison. The interviews, the committee report and the inter 
report to the board are good indices for local reaction to the project. In 
addition, by identifying certain administrative problems — priorities, flow 
of information, channels of communication, data quality and public relations - 
the picture of a large school system engaged in innovative efforts becomes 
real. 

J 

Summary 

\ 

To sum up Part II, these conclusions bear repeating. First, performance con- 
tracting programs are not pre-packaged, they are in the developmental stage. 
Consequently, their implementation and modification have occasioned many 
problems. Second, cognitive gains,. as measured by standardized achievement 
tests, were "respectable," despite the large room for improvement in the area 
of testing. Third, costs compared favorably with those of Title I remedial 
programs. Fourth, evaluation procedures and data were unsatisfactory. Fifth, 
management posed some real sticky problems in the operation of performance 
contracts. Sixth, profit margins have generally been low, forcing many con- 
tractors to shift their role to that of consultants*. 
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Part III - Performance Contracting Guide 

The third part of the Rand Report, dated March, 1972, is a guide to performance 
contracting. Its authors were Hall, Rapp, Carpenter, Sumner. and Haggart. The 
guide is meant as a kind of handbook for school board members, administrative 
and other decision-makers who may be considering performance contracting. 

Volume 9 deals with the legal issues, program management, measurement and 
validation of performance contracts. In general, it raises important ques- 
tions and implications, while pointing out the special requirements made of 
schools as a result* 

The guide makes a special point of addressing the legal issues associated 
with performance contracting. (Some problems were raised in the Gary volume.) 
The authors express concern that the LEA establish its authority over a given 
project, that it comply with state procurement regulations in the selection of 
an LSC, that it observe mandated instructional time allotments, that it use 
certified textbooks and materials, that it insure teacher quality and condi- 
tions of employment according to existing agreements (regarding negotiation, 
clasri, size, assignment and transfer of teachers, and teacher compensation), 
that it make provision for student rights,' and that it secure legal protec- 
tions in the form of warranties, bonds and insurance. This sort of legal re- 
search then is the first step in the operation. 

The second step requires that a school and its community assess their needs 
and then define their goals and objectives. At this point various kinds of 
management support are available to help an LEA initiate and implement an 
innovative program. The authors caution, however, that such assistance is 
expensive. Moreover, it urges that teachers be involved at an early stage 
(an oversight in the GEO experiment). 

The Rand team delineates the different purposes of evaluation; to measure 
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results, to validate/certify results, to aid in decision-making, and to 
assist in improving programs. They then summarize the types of instru-- 
ments that can be used to accomplish one or all of thf^^ purposes of an eval- 
uation design for a so-called ^^quasi-experiment^' — standardized testsr 
measures of cost effectiveness (e.g. Webster and Blaschke), criterion-refer- 
enced tests, interviews, direct observations, attitudinal studies of parents 
and teachers, and control groups. 

In the area of testing, the guide looks at certain logistical and administra- 
tive problems. It also spells out basic test functions: to diagnose, to 
demonstrate mastery or achievement, to determine payment to the contractor, 
and to evaluate a program. Much has indeed been learned about testing through 
the whole experience of performance contracting^ The obsession with preventing 

teaching the test" seems to have passed, not, however, without effect. 
(Penalties were devised that an.cunted co Si, 000 per test item or o immediate 
termination of contract. A series of approaches to "mask or disguise" a test 
was developed, including simply blanking out the name of the test or else 
retyping it.) 

To select an LSC, the school is advised to use the competitive bid pro- 
cedure, rather than , the sole-source method. The contract proper should 
stipulate an amendment procedure as well as definite contract settlement 
procedures. (From the OEO experience, forewarned is forearmed) • 

To organize and monitor programs makes certain demands upon a school. It 
must develop a rationale for selecting participating schools, personnel 

and students; for training teachers before and during the program; for main- 
taining records; and for "extending awareness of the program." (emphasis added) 



Volume 10 deals with the more technical aspects of performance contracting, 
especiall;y in connection with cost analysis. The attempt is to develop 
a measure of cost-effectiveness. Sample contracts from Volumes 3-8 fill 
out this report, producing in effect a rather comprehensive, yet brief 
and manageable guide. It is unfortunate that such material was not avail- 
able for the planning of the 1970-71 projects. 
Analysis of the Rand Report 

Certain comments need to be made on the Rand Report on educational perfor- 
mance contracting. As Mecklenburger has already pointed out. Part I is 

rather lifeless — "a strangely dispassionate and obtuse do>oument." By 
ignoring the specifics of individuals and social milieu, the report only 

weakly explains the genesis and growth of performance contracting. Second, 

the references are few in number and wide-ranging; they are inadequate for 

the purpose of documenting and describing both the operation and theory 

of performance contracting. 

Part II is a virtual treasury of performance contracting documents. It 
contains, for example, an RFP (Request for Proposal), 2 proposals, a lettev 
of intent, material lists, 9 LEA-LSC contracts, 1 MEA-MSG contract, k LEA- 
Evaluator contracts, and 2 LEA-Auditor contracts. In addition, it has a 
committee report and an interim report to the board of education. These 
documents constitute almost half of Part II, and as such serve as a resource 
for further study, especially for comparative purposes. 

Part II", however, is basically an analysis of 8 performance contracts (with 

footnote reference to some 20 others), of 7 LSC's, of 15 schools, of 6 

school districts, of 5 cities, of 2,500 kids and of 112 teachers. By focusing 

mainly on this sample of non-OEO projects, the report is necessarily incomplete 
in and of itself. 



The studies are essentially descriptive analyses. They do not represent 
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evaluation in the broad or even narrow sense. Also, as a synchronic approach, 
the report has narrow limits ~ 1970-71. To add depth to the analyses, a 
follow-up return visit has been scheduled, to establish longer raJige observa- 
tion. 

The cost analysis using the standard replication cost model is unconvincing, 
largely because it is not based on complete figures of actual cost. It aJso 
fails to take into account the low cost components of many experimental pro- 
grams — lower paid, energetic personnel. To base a cost comparison on these 
data is risky business, since the program naturally becomes more expensive as 
more experienced personnel are engaged for higher salaries, with less 
inclination to provide "free resources.'* The argument for lower costs be- 
cause of the use of paraprof essionals likewise fails to consider the inevitable 
improvement of their conditions of employment. 

Finally, it should be noted that the report as a whole has an extremely 
useful format. Other analyses could well take example from the division 
of a report into attractive 9 readable and orderly parts of reasonable length. 
The report bears many of the marks necessary to serve as a tool for decision- 
makers. 



.CHAPTER IV 
LOCAL SCHOOL DISTRICT EVALUATION REPORTS 



A general assessment of performance contracting can be made in a number 
of ways, ranging from sobering statistics to inflated promotion. Companies 
have withdrawn from the field to the point that, according to a reliable 
source, only one company may have a real interest next year. Thf; running 
total of performance contracts has fallen off markedly* Meanwhile, the 
heat that accompanied many projects has dissipated, causing conferences 
to search for hotter issues for discussion. Finally, the horizontal and 
vertical mobility has caused many key figures to abandon the scene, leaving 
ghost projects behind. As a result, the game is still in play, but the 
subs use a different style. And the outcome may be different as well. 

Evaluation reports like performance contracts defy generalizations. They 
have not yet assumed a standardized form. In this way, the peculiarity of 
each project is manifest in its conditions, participants, time and place. 

The following conclusions are based on an analysis of over 25 evaluation 
reports, over 28 personnel communications with project personnel, a question- 
naire returned by 48 State Educational Agencies (SEA's), conference events, 
and visits to United States Office of Education (USOE) and the Office of 
Economic Opportunity (OEO). The Rand and OEO reports have also been 
examined, but are treated separately in this study. Hence, empirical 
evidence supports subsequent observations. 

From single page impressions to bulky statistical tables, evaluations are 
alike in their general lack of common and clear purpose. Their writers, 
however, face certain difficulties. The readership no longer encompasses 
just a supervisor or administrator, but school board members, SEA and USOE 
officials, press agents and researchers of every decripticns as well. 
Additionally, the reports are potential propaganda, to be used on prospective 
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customer's, by schools, contractors and individuals alike. In aiming at all 
audiences, the reports seldom hit the mark. 

The evaluations attach special significance to data, as a rule. Sometimes 
they seem unconcerned about what the figures stand for, or else immodest 
about flashing around percentages and totals. The end result is a kind 
of *'data manipulation," as Roscoe Smith from the Dallas Office of Accounta- 
bility jokingly referred to it. Nonetheless numbers talk, and lend 
statistical support in the form of test score differences, attendance 
figures, attitudinal scales, lists of materials and total dollar amounts. 
Unfortunately, the drta were often gathered indiscriminately and yield at 
best inconclusive interpretations. 

The foregoing is presented not to demean the evaluation reports, for they 
are virtually the only coded source of information available on the local 
scene. The intent, rather, is to warn against possible misuse of them, 
under the false assumption that generalizations with numbers are not only 
easy but sure. 

Another critical factor in evaluation is timing. Upon their appearance 
reports on evaluation are obsolete and inconsequential. As post facto ■ 
recreations, they attest the fact that a decision has already been made, 
and the report is accordingly "prettified" or filed away summarily, 
Behind it all lies the operating principle that decisions to Continue or 
to suspend are based on factors other than formal evaluation reports or 
their contents. 

In so far as they succeed in documenting the implementation and process 
of performance contracting^ evaluation reports assume usefulness. Moreover, 
as the final, tangible evidence of a project, they outlive the conditions 
and personnel. What they preserve, then, however accurate, becomes the truth 



- Hi - 



in time. And so history is rewritten. 

V/hat then does this extended discussion of individual evaluation reports 
mean? The answer is a pessimistic shrug of the shoulders, admitting we 

just don't know much about performance contracts, in spite of the reports. 
At least we can't tell from those reports unless we approach them in 
credulous fashion and accept a pat answer. 

Michigan State Superintendent of Public Instruction John W. Porter expressed 
the situation well: "Performance contracting will be effective, operational 
and successful only to the extent that the individual state and school di- 
trict want it to be."-^^ 

It is an oversimplification to say that a program did not work without 
going into the reasons why. It may have been the personnel who were re- 
sponsible for the outcomes. 

Some insight into the effects of and reactions to performance contracts can 

be gained by sampling the correspondence of school officials and project 
personnel, (emphases added) 

"Because these reports were so voluminous , we have not repro- 
• duced them for distribution but they are on file in the district 
for anyone who wishes to pour, (sic) over the contents." 



"With regard to gains per dollar, it would require copying some 
200 pages to include all data from our model. The model is available 
to your staff if anyone is interested ." 

"We undertook another deal using our own teachers and materials but 
contracting with an outside firm for in-service education of 
teachers. That program seemed to succeed. . . ." 

"As you know x has moved from using the term performance con- 
tracting to contract learning. In light of the recent OEO 
results we have now changed our tack to individualized education." 

"The report speaks for itself. The only thing it doesn't say is 
^^^^ X owes money to the school department." 

"The OEO performance contract . . . was quite unsuccessful and in 
fact the performance experimental group was in many ways inferior 
to that of the control group." 
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"We did as a district fund a limited experimental program 
utilizing the LSC materials; however, we have not prepared a 
report of the character that would be comparable to the major 
performance contracting experimentsn" 

"We had a performance contract with x which has we feel 
been highly successful." 

"The OEO program ran for the year I97O-7I and has since been 
phased out ." 

A second type of constraint may surround the evaluation of a performance 
contract,, As a result, the report may not be available, as suggested by 
the following excerpts: (emphases added) 

"As you know, the circumstance surrounding release of information 
on performance contracting have been less than ideal. x has 
retained the right to release information on p'^rformance contracting. 
Although this he apparently been done on a national level, the 
necessary information to complete our local evaluation has not been 
received^ nor has the authorisation ." 

"Internal performance contracting is an integral part of x * s 
differentiated staffing project. It has not been separated for 
any evaluation purposes, however. Any information or quasi-eval- 
uation data is as a component within the differentiated staffing 
model." 

". . .our involvement is very much bound by x constraints. Access 
to the reports given to the Office of Education by these two corpor- 
ations may or may not be possible ." 

"It would be premature for me to speculate on the results, but I 
promise you a copy of the evaluation report, assuming the school 
dj.stricts concerned concur , and a copy of oar contract model." 

"While our office does not have the contract, we do have the proposal 
regarding the laboratories, but I am not allowed to release it ." 

"It. is the Division's policy (USOE) not to encourage wide distribu- 
tion of the reports or to indiscriminately pass out copies." 

Question at AERA Conference: "Would you care to comment on the 
reasons for the cost overrun in the OEO evaluation contract or the 
reasons for the delay in the completion and release of the OEO 
report?" (Author) 

Answer: "No comment — the schedule was deviated from." (Director, 
OEO-TAC contract) 

Evaluation reports, therefore, have to be read in light of the above-mentioned 



factors. 



CHAPTER V 



REVIEW AND ANALYSIS OF OFFICE O F ECONOMIC OPPORTUNITY EVALUATION REPORT 

The report by the management support contractor (MSG) for the Office of 
Economic Opportunity (OEO) experiment — Education Turnkey Systems, Inc. — 
is a source of much practical experience. Released prior to the announcement 
of achievement data, it sketches well the problems encountered in imple- 
menting and administering "the largest single nationwide experiment in the 
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history of public education." The MSG cites a twofold purpose to OEO*s 
use of performance contracting: l) as a vehicle for evaluating the cost- 
effectiveness of six different instructional learning systems in mathematics and 
reading, and 2) as a technique of instruction in mathematics and reading. (Emphasis 
added) The OEO assumed responsibility for conducting a low-cost, low-risk 
evaluation of learning systems, with an orientation toward outputs, with 
minimal administrative costs due to management decentralization with quality 
control, and cross fertilization effects. 

Problems Which Hindered the Evaluation 

Initially, the MSG identified a slightly different purpose to the exper"*ment 

from the one reported by OEO: "The overriding objective of this experiment 

was to determine what types of instructional systems work best Wxth 

what types of students in producing achievement in mathematics and reading as 
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measured by standardized tests and IPO's (interim performance objectives)." 

Next, the MSG cited administrative problems connected with the experiment. 
Contracts, for example, did not spell out the responsibilities of the school 
district, which were rather assumed in "good faith."' (lilsewhere, Blaschke 
commented that 10-rll of the contracts could have been terminated for Local 
Educational Agencies (LEA) non-compliance. This ^is a product of the 



'^experiments come aiid go'^ syndrome.) Moreover, the OivJ-LKA contract lacked 
ince]itiveG and consequently limited the level of quality control possible. 

Thirdg foderal : r. Lerv^'ntior in local schools had contrary efffectr>. Some 
LEA's, LSC'e^ (learning services contractor) and parents objected to OEO's 
meddling in local affairs and assuming complete control. (This might be called 
the "Washington" syndrome.) At the same time, by requiring the permission 
of parents to involve their children in an experiment, federal influence 
causes greater responsiveness to local conditions. Unfortunately, the 
responsiveness was not always productive. For example, OEO's definition of 
a target population for purposes of experimentation appeared to run counter 
to previous desegregation efforts in the instance of Taft, Texas, where 
parents opposed a 90% Mexican- American composition in the experimental group. 

A final administrative difficulty was the matter of tracking highly mobile 
and truant students. These special problem students hampered the operation 
of the LSC instructional progran:. 

The late selection -^f a testing and analysis contractor (TAG) — Battelle 
Memorial Institute — brought about some conflict over roles. Initial ro- 
sponsibilities for documentation were assumed by "ETS" , but later were 
shifted to Battelle. Evaluation efforts were also hindered by the managerial 
flexibility that the LSC's enjoyed. They were encouraged to use whatever 
worked. Finally, a certain friction developed between OEO and LEA's over 
the rigor of the evaluation design, which largely precluded local invclvenrent 
in decision-making. 

The MSG interprets " the lack of sojie other potential problems in the OEO 
experiment to be the result of the waiver of state and local constraints. 



In the same vein, it advises: "Soan i planning and decison-malcini^ based upon 
education rr.erit will lead ^.o education reform more quickly than prescriptive 
or proscriptive legal tenets." 

Recounting the legal incidents that took place, the report lists the filing 
of a grievance by a teachers' group over bonus payments to teachers, a 
lawsuit filed by teachers at one site (dismissed in mid-course), an injunction 
over the lack of certified teachers,, a boycott on charges of segregation 
and State Education Agency (SEA) threat to terminate the contract (whereupon 
a ruling by the state attorney general upheld the federal contract which 
by-passed the SEA). Specific matters of instruction that evoked controversy 
included the use of paraprofessionals, teacher incentives, student incentives, 
logistical problems and staff turnover. 

Certain changes occurred in the instructional process itself. The teacher, 
who traditionally had been a source of knowledge, became a "manager" of 
learning and resources. His/her duties came to include diagnosis and pre- 
scription, Blaschke labeled this phenomenon as not only learner-centered, 

but learner-controlled instruction as well. He observed, however, that 
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this practice could be abused at the secondary level. (Saretsky makes 
the same point in "Every Kid a Hustler".) In actual fact, the junior high 
students in Taft, Texas, objected to certain reading materials, with the 
result that the course was changed to their satisfaction. 

Certain problems developed between administrative personnel, specifically 
between the project administrator (LSC) and the project director (LEA). 
These interface problems, at times, might have been the result of bringing 
in "outsiders" for^ purposes of management. Within one LSC, two changes 
in center managers jere made during the course of the program. 



Lear uinf?: Support Mechanisms 

The M.^G report mct}>:eL5 an inventory of the various incentives ased, without 
howe"^-r, J^/ '^Lnii nii precisely v/hat effects they had. The effectivenss in 
the J'7-iri.iing --'^C:..:.? is documented (Educational Technology Goiifereuce — 

NYC). The inc^^ntives in the OEO experiment included points redeemable in 
catalog stores; money (play and real); and gift items such as dictionaries, 
telescopes, .free time, credit. cards, tickets, certificates, and emblems^ 
In short, the extrinsic rewards mirrored the present society. Other firms 
stressed traditional instrinsic rewards — the "heaven'^ approach, where 
you work now,' reap your reward later, whatever the i' orm • This contingency 
management measure (incentive) v;.-=^s designed: l) to reward achievement, or 
2) to influence behavior such as attitudes, attendance, speed, and compre- 
hension^ ' • 

Some performance contracts involved subcontracts betv/een the student and 
the teac.lier. Students were encouraged to take the initiative and determine 
their goals, then scht^dule their tasks to achieve those goals. In some 
projects, tutor-ing vv^^s employed both singly anu in ..jiLal] -^^roups. 

Test Results 

The final portion of the MSG report is devoted to a cost analysis and com- 
parison between experimental and control groups. Lacking achievement data, 
the analysis is based on costs only and does not provide a cost-effective 
measure. The analysis fails to take into account the incentive costs for 
teachers and LSC .personnel and the associated administrative costs. Partly 
for this reason, project directors. have questioned the validity of the 
approach. , 

The cost analysis focuses on grades y and 8 as representative of the elementary 
and secondary programs. The rationale for the analysis rests in the consump- 
^ ^io^ resources — . thoir costs and a prorated cost for support services. 

EPJC 

mamsim All in all, the results are based on the .examination of 12 of the control 



schools and 10 of the experimental schools. A shortcoming of the approach 
is the unit — costs per student per student year — where student year is 
not actual time in minutes, but is average number of days of attendance. 
The method does not differentiate amjng courses with unequal time allotments 
per day as would the cosb per student achievement gain method suggested 
by Jokn Webster. 

Test Results Summarized 
The following Listing presents a summary of the test results as reported to 



the individual project sites in the OEO experiment. 



SITE . 




REPORTED EFFECTS 


MEASURED 


EFFECTS 








SIGN DIFF+ 


GAINS* 










E C 


Seattle, W/i 




very poor 


11 


1 6 


Selmer, TN 




slightly positive 


5 


3 0 


Las Vegas, 




unsuccessful 


3 


1 k 


Dallas, TX 




somewhat better 


3 


6 0 


Bronx, NY 




poor (grades 2 & 3) 


1 


0 2 


Athens, GA 




somewhat better — ■ 


9 


8 1 


Jacksonville , 


Fl 


somewhat faster/no difference 2 


6 0 


Hammond, IN 




little difference 


^ 11 


0 1 


Portland, ME 




quite poor 


10 


0 6 


Hartford, CT 




poor 


2 


1 3 


McComb, MS 




little difference 


10 


2 1 


Rockland, ME 




slightly positive 


12 


5 0 


Anchorage, AK 




positive 


10 


7 0 


Ph \ladelphia, 


PA 


poor 


6 


0 5 


Taft, TX 




slightly positive 


7 


2 1 


Mesa, AZ 




no significant difference 


2 


2 2 


Stockton, CA 




very little difference 


1 


. 2 1 


Fresno, CA 




poor 


7 


0 5 


Wichita, KA 




little difference 


7 


1 1 


Grand Rapids, 


MI 


no data 






+ number of pre- 


■test significant differences 


in favor of control group 



out of 12 grade/subject combinations 
* number of post-test significant differences in favor of E (experimental) 
or C (control) group out of 12 grade/subject combinations 



The above results overwhelmingly condemn performance contracting in terms of 
the subjectively reported effects (which ranged from very poor to positive). 
The measured effects, however, do not provide evidence to support such a 
blanket denigration of performance contracting. First, the categories used 



to establish the differences between the control and experimental groups 
are highly arbitrary. Second, there is no accounting for significant 
pre~tee:t difference;: betv/^eri i:h' rronp:^. Third, on the basis of the number 
of significant differences between the gains of the two groups no definite 
conclusions may be drawn. Fourth, judgments were based on grade equivalents. 

It is immediately appai'ent from the number of significant pretest differences 
that there was a mismatch on achievement in favor of the control group (in 10 
sites a majority of the 12 grade/subject scores utilized favored the control 
group) . 

This report on test scores made at least two questionable assumptions. First 

is assumed that adjustments for initial pre-test differences between groups 

is possible (see Klein paper in the appendix for criticism). An example of 

this assumption is the analysis of Hammond; 

"Since the control students started significantly ahead in every 
case but second grade reading, the mean gain comparisons under- 
estimate the relative experimental gains by about 1-2 raw score 
points." 

Second, while documenting pre- and post- test conditions, the report claims 

to be able to adjust for poor testing conditions or at least to gauge the 

effects, as in Hartford: 

"While some pre-test testing problems were reported, they appear 
to have affected both groups equally, " 

The reading on Dallas reflected the same questionable assumption: 

"Pre-test testing conditions in Dallas were poor in the upper grades 
for both experimental and control students. It is conceivable there- 
fore, that although there were no real differences among the control 
and experimental students, poor testing conditions affected the 
seventh grade experimental group less adversely than the seventh grade " 
control group and vice-versa in the eightii grade." 
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Third, the conclusion oi^ Portland is just plain inaccurate: 

"Although the experimental group stai^ted significantly behind the 
control group in every grade and subif^ct. except second n;rade uiathetnntics 
and ninth tiTade reading, the dif forcj-.c^;. were not ver,y "Teat." (10 signi- 
ficant differences!) 



Finally, the report of test scores is simply an interpretation that lacks 

persuasion, as in Jacksonville: 

^^Students in the experimental group in Jacksonville seem to have 
achieved at a somewhat faster rate than the controls in the lower 
grades, but performance contracting seems to have made little dif- 
■ference in the upper grades." 

The same result could have been reported in reverse order: 

Performance contracting seems to have made little difference in 
the upper grades, but students in the experimental group seem to 
have achieved at a somewhat faster rate than the controls in the 
lower grades. 

Obviously, it depends on the* point you- are tjrying to maJce. Anchorage is 

another case in point: 

"Performance contracting in Anchorage showed a positive effect. The 
experimentals gained more than the controls in every grade/subject com- 
bination except in eighth grade mathematics, where there was no signi- 
ficaint dii ference between the two groups. 

Obviously there was a good bit of editorializing in the reported test results • 

MSG Summary Statement 



In summarizing the results of the experiment, the MSG cites as a major 
accomplishment the fact :hat all of the twenty school districts, six firms, 
and two teacher associations under contract when the project began remained 
in the project throughout its duration. In the wake of the Texarkana test 
teaching scandal, the highly structured OEO experiment showed: "No indication 
of teaching to the tests was uncovered. In that respect, the project 
O should have been recognized for proving the effectiveness of safeguards- 
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The MSG su^^-ests some recommendations on the basis of the monumental nation- 
wide experiment. First, LEA's should have greater input into the selection 
of schools and students (not by decree). Second, the requirements for the LEA 
representative, the project director, should be more specific about desired 
experience and capability. In some cases he was engaged at a rather late 
date, impeding the implementation of the contract. Third, roles should be 
defined more clearly, especially with regard to data collection and documen- 
tation. A number of obstacles were encountered in this area, due to: 
1) late selection of TAG (August 21, 1970), 2) school district non-compliance, 
3) split responsibilities of analysts engaged in doctoral research, 4) extreme 
variation in sophistication and capability of data reporting systems of 
schools, and 5) excessively detailed documentation. 

To improve the stability of the student groups, they could be selected on 
the basis of pre-test scores rather than "baseline data," and two-thirds of a class 
could constitu':e the experimental group, with the remaining one-third serving as 
replacements. This would eliminate a number of the problems associated with 
these procedures during the experiment. 

GEO Preliminary Report 

The OEO Summary of Preliminary Results (February,, 1972) has become a source 
of considerable controversy. This docunent, released at a Washington, D.C., . ^ 
press conference, armed the outspoken critics of the OEO Performance Incen- 
tive Remedial Education program (later simply called "Performance Contracting 
Experiment"). It "flunlced" performance contracting as conducted in 1970-71^ ' 
At the same time it caused advocates to stiffen in their resolve as they were 
backed into a corner. 

At the press conference, OEO officials "reiterated" the objectives of the 
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experiment: 

"to t?st whetb'^r a representative group of private contractors with 
exicting techniques operating under performance-based contracts could 
improve reading and mathematic skills of poor children,''^^ 

Why was the experiment undertaken? 

- to help poor children 

- attractiveness of concept and its emphasis on outputs 

- favorable reports from Texarkana 

- indications that performance contracting would become a fad 

On the other hand, the experiment did not- intend to: 

- provide a consumer's rating of various contractors 

- determine which educational technique was best 

- develop new education programs 

- measure precisely the effect of incentives on contractors, teachers 
or students 

The experiment was intended to provide information that educators and school 
boards presumably need relative to performance contracting. The program was 
launched in May, 1970, shortly before the Texarkana scandal hit the deadlines. 
Results were originally slated to be released in the fall of 1971 i which 
encouraged an attitude of "wait-f or-a-year-and-see OEO hoped to subject 
new instructional technologies to a "rigorous evaluation," capable of 
generating information with "broad applicability," 

Two aspects of the original objective of OEO*s experiment can be challenged. 
First, as Loyd Dorsett has maintained, the LSC*s selected were quite similar 
to one another, with all but one basically soft-ware oriented. In his esti- 
mation, there was a serious lack of representation of hardware-oriented approaches- 
audio-visual teaching machines, television or computer-assisted-instruction. 
As Martin and Briggs* stated "Most students end up using programmed workbooks," 

Second, the experiment was "to evaluate the relative effectiveness of existing 
techniques, not to underwrite the development of new techniques." It seemed 

O quite clear from the Texarkana operation that, in the demonstration stage. 
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an LSC had to develop its program to make it work. The materials hnd to 
be adapted to trie grade level and particula-r capability of each student. 
Later, the Rand report confirmed the developmental nature of instructional 
programs in performance contracts. 

Prefacing its conclusions, OEO explained the basis for its generalizations: 

"The broad conclusions that are outlined here can be viewed with confidence, 

but idiosyncracies concerning sample characteristics, testing conditions, 

and other factors necessitate that caution be used when results for individual 

^8 

sites are examined. The rationale for this approach is specious, par- 
ticularly in view of Battelle*s warning against interpreting aggregated 
results. 

One of OEO*s basic assumptions in measuring the results of the experiment was 
that success on standardized tests strongly related to general succcess in 
school. Therefore, achievement results would be generalizable, in their view. 

In the area of testing, the OEO experiment was strongly influenced 
by events in Texarkana. Strict measures were taken to enforce test security; 
a strong penalty clause was prov^ided (termination); LSC personnel were for- 
bidden to administer or to score tests or to find out their identity; a: 
curriculum audit was scheduled; and provision was made for retention tests 
to be given the following year. Because of this great concern for testing 
operations, communication gaps developed in the face of classified information 

OEO scheduled a variety of standardized tests for payment and for evaluating , 
purposes. Initially high on criterion-references t^sts, OEO was unable to 
mount a successful testing program on the basis of item pools and consequently 
CHine to play down their significance: "It seems that some of the tests were 
too easy." In fact, it turned out that the TAG (testing and analysis contract 
did not review the tests before they were given; instead it certified them 
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afterward if they were okay. Although this matter is handled delicately 
in OEO and BMI reports, it is rumored that the TAG refused to certify 
a number of tests. OEO, in this report, admitted the sorry outcome: "Thus 
the IPO (interim performance objectives) appear to have been virtually 
useless for evaluation purposes and to have had questionable value for 

ZiQ 

payment purposes." 

What results did OEO report at the press conference? To the question: "Was 
performance contracting more successful than traditional classroom methods 
in improving reading and mathematics skills of poor children?" They replied: 

"No." ". . .performance contracting was no more effective in either reading 
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or mathematics than the traditional classroom methods of in-^truction." 
( emphaj3Ls added ) 

While acknowledging, on the one hand, that the' "students in the control 
group did unexpectedly well," they proceed to generalize: "Yet, in fact 
neither group did well." ("well" here means apparently achieving a one 
year grade level gain, attained by two of the twenty sites. )^"^ Charles Stalford, 
project manager of the OEO experiment, conceded that a "horse race" was on, 
with but little control over the "control" group. He added that while OEO 
had little actual control over test quality, testing was a "bane" from the 
start. 

The OEO results were even more disappointing from another point of view — 

their relation to grade levels. It turned out that the experimental group 

slipped even further behind during the year. The point is hammered home: 

"Performance contracting was not responsible for any significant improvement 
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on an overall basis, "-^ (It turned out that OEO had its own definition of 
"significant" — namely, .5 grade level gain.) 

Looking for individual successes, OEO checked the 20ib ,40ib, jotb, 60^ and SOtb 
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percentiles, and concluded: "There is no evidence that performance contracting 
had differential results for the lowest or highest achieving students in the 
s<miple.''^"^ (Still the registered gains ranged from .k to .8 grad-e equiva- 
lents.) 

) 

Finally, elaborating on its conclusions, OEO explained '*that there were no 

significant differences in the achievement gains of E and C groups." ' ' - 

Further, "the most interesting aspect of these conclusions is their vex-y 
55 

consistency." 

Returning once again to the original objective of the experiment, it is in- 
teresting to note that while performance contracting is expected to improve 
skills, no reference/standard is given. The question naturally arises: 
Does "improve" mean to do better than the control group, or to do better 
than in previous years? Seemingly, OEO applied the first meaning, and 
failed performance conctracting for not doing any better than "traditional" 
methods. 

In relation to what seems to have been a major concern for OEO — indications 
that performance contracting would become a fad — the summary concluded 
patently: 

"On the basis of these findings it is clear that there is no 
evidence to support a massive move to utilize performance con- 
tracting for remedial education in the nation's schools. School 
districts should be skeptical of extravagent claims for the 
concept. "56 

The intention may have been to halt an educational fad by concluding: "The 

results simply say than an uncritical rush to embrace these concepts is 

57 

unwarranted at this time." 

Concerning the problems of poor children, OEO admitted: ". • .we still have 
no solutions to the specific problem of teaching disadvantaged ^roungsters 
basic mathematics and reading skills." "The search for solutions to these 



problems must continue.'* By such a clear and outspoken conclusion, OEO 
at least tacitly gave support to the theory that educational problems are 
unrelated to social and economic conditions, as the Coleman Report convinc- 
ingly suggests. The basic problems may be more than mathematics and reading skills. 

Basic Gain and Cost Data 

OEO^s. summary provides important cost information on the various proposals 

of the six LSC^s. Their prices ranged from $l85 to $2^ per student per 

subject per year ( in keeping with the national average of roughly $200). 

This was done to facilitate the possibility of replicating the programs in 

any school in the country. Minimum gain guarantees ranged from .5 for grade 

1 and 2 to 1.5 for grades 7-9 as shown in Table I. 

TABLE I 

MINIMUM ACHIEVEMENT LEVELS IN READING 
AM) MATHEMATICS GUARANTEED IN OEO 
PERFORMANCE CONTRACTING EXPERIMENT 



GRADE A IF P QED S/G WLC 

r mrm rmrm rmrm 



1 * .8 7E TTo TTo 73 T3 TTo O 73 73 iTo TTa 

2 * .8 .8 1.0 1.0 1.0 1.0 1.0 1.0 .5 .5 1.0 1.0 

3 * .8 .8 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 i.o i.o 

7 1.0 1.0 1.1 1.1 1.0 1.0 1.5 1.5 1.5 1.0 1.0 1.0 

8 1.0 1.0 1.1 1.1 1.0 1.0 1.5 1^5 1.0 1.0 1.0 1.0 

9 1.0 1.0 1.1 1.1 1.0 1.0 1.5 1.5 1.0 1.0 1.0 1.0 

- .5 at Taft 



Note: As used in this section. 



r= leading 

m= mathematics 

E= experimental group 

C= control group 

G.E Q./g.e.= grade equivalent 

A~ Alpha Learning Systems 

P= Plan Education Centers 

LF= Learning Foundcitions, Inc. 

QED= Quality Education Development 

S/G= Singer/Graf lex, Inc. 

WLC= Westinghouse Learning CorD. 

BRL= Behavioral Research Laboratories 

COMGS= - Combined Motivation and Education Systems 
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■•^•^om these tables, it is apparent that grade level c'^i^i^'^ must be more difficult 
to achieve in the lower grades since the guarantee level rises in both 
rtirading and nicithem/AUics in the Lii|-:lier grades, lb is of :;onie intero:.:t to noLe that 
minimum guarantees for reading and mathematics are identical at all levels for 
each company. 

Similarly, it is worth examining the costs per student per grade level gain per 
subject. For example, how much does it cost each company to produce a gain 
of one year for a third grader in mathematicsV The prices per grade level 
gain are listed in Table II. 

TABLE II 

. ■ PRICE PER STUDENT PER GRADE PER SUBJECT FOR 
ACHIEVEMENT GAIN OF 'ONE GRADE EQUIVALENT 



GRADE A LF P ^ QED S/G 





r 


m 


r 


m 


r 


m 


r 


m 


r 


m 


r 


m 


1 


70 


70 


101 


101 


92 


100 


72 


72 


165 


165 


75 


75 


2 


70 


70 


101 


101 


hG 


50 


72 


72 


165 


165- 


75 


75 


3 


70 


70 


101 


lOi 


hG 


50 


72 


72 I 


82 


■ 82 


75 


75 


7 


75 


75 


7h 


7h 


55 


50 


55 : 


55 


82 


82 


75 


75 


8 


. 75 


75 


7h 


7^ 


55 


50 


55 


55 


82 


82 


75 


75 


9 


75 


75 


7h 


7h 


55 


50 


55 


55 


82 


82. 


75 


75 



(all figures rounded off to nearest dollar) 



As Table II shows, the LSC price per grade level gain generally decreases from 
grade 1 to grade 9- However, Ran's prices are interesting in that they 
show a higher cost per grade level gain^in mathematics in the elementary and higher 
in reading in the secondary. 

Finally, as shown in Table III, .1 grade level increase above the minimum 
guarantee would seem to reflect the motives of the LSC^s — i.e. where they 
expect to make tneir money. 
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TABLE 111 

PRICE PER STUDEMT PER GRADE PEK SUBJECT 
FOR EVERY .1 GRADE EQUIVALENT GAIN . 
ABOVE MINIMUM GUARANTEE 
ACHIEVEMENT LEVEL 



A LF P QED S/G WLC 





r 


m 


r 


m 


r 


m 


r 


m 


r 


m 


r 


m 


1 


6 


6 


9 


9 


9 


20 


9 


9 


9 


9. . 


11 


■ 11 


2 


6 


6 


9 


9 


9 


20 


8 


8 


8 


8 


11 


11 


3 


6 


6 


9 


9 


9 


20 


8 


8 


7 


7 


11 


11 


7 


5 


5 


8 


8 


6 


10 


15 


15 


7 


7 


11 


11 


8 


5 


5 


8 


8 


6 


10 


15 


15 


7 


7 


11 


11 


9 


5 


5 


8 


8 


6 


10 


15 


15 


7 


7 


11 


11 



(all figures rounded off to nearest dollar) 



Except for one firm, LSC's stood to gain more per .1 grade level in gain 
in. the elementary than in the secondary. Plan again either anticipates a' 
greater difficulty in teaching mathematics than in teaching reading or expects a 
greater profit from math than from reading. 



Interim Report from TAG (Bat telle) 

Interpretations of the TAG report — The Office of Economic Opportunity ; 
Experiment in Educational Performance Contracting (January 29$ 1972) — 
served vas the substance for the OEO press conference discussed earlier. 
This interim report by the TAG — Battelle Memorial Institute — anticipates 
its final form, except for its single method of analysis. This pre-post 
method of analysis is limited to measuring gains for roughly two-thirds of the 
students in the experiment. The others, who did not complete the program 
or missed one of the tests, were not included (until the final report). 
Contrary to other reports, TAC's analysis was done. with raw scores. Grade 
equivalents were used only for the convenience of reporting (*'a commonly 
used metric^'*) 

O „ To determine first grade results, a readiness pre-test was compared with a 

ERJC 

^" """" first grade reading post -test. (Webster, Hall and Blaschke criticized the 
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use of first graders in the experiment.) Nevertheless, the TAG reported 

the following results: (pp 76-85). 

G rade 1 - '^Overall, positive differences in post-test means occu^ 
in 5 out of 8 sites in reading, and occur in 3 out of 6 
sites in mathematics." 

Grade 2 - "The table shows 5 positive and 7 negative differences in 
post- test means in reading; and 2' positive and 7 negative 
differences in mathematics." 

Grade 5 - "The results show 5 positive and. k negative differences 
in the post- test means for reading; and 3 positive and 8 
negative differences in the post-test means for mathematics." 

Grade 7 - "In reading there was one positive and one negative difference 
in post-test means. In mathematics all k differences were 
negative." 

Grade 8 - "Three sites out of 8 showed positive impacts, in reading; 

none of the 6 sites showed positive impacts in mathematics." 

Grade 9 - "In both reading and mathematics, 5 of the 6 showed positive 

differences in the regression estimates of the post- test means." 

It is noteworthy that the conclusions were based on sites showing significant 
■group differences. Results summarized below indicate the number of signi- 



ficant differences by grade 


level for reading and mathematics 


Grade 


r 


m 


r' 


m' 


Low pre-post correlation 


1 


8 


6 


6 


6 




2 


10 


9 


7 


8 


.50 


5 


7 


15 


k 


11 


.50 


7 


2 


k 


2 


5 


.60 


8 


8 


6 


5 


5 


.60 


9 


6, 


6 


5 


5 


.60 ■ 




kl 


kk 


29 


58 





' - excluding tests with low pre-post correlation 

Out a total of 212 possible grade/site/subject combinations, the 
above refers to a total of 85, or 67 when tests with a low pre-post cor- 
O relation are excluded. It is clear that these cases comprise less than half 
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of the total number of 212. 



Finally, the TAG summarized its preliminary findings in a chart that pointed 
out instances where the results between the control and experimental stu- . 
dents were significantly different. These differences were labeled '^positive 
impacts" or "negative impacts" according to their relation to the experimental 
group, 

. .Five positive impacts were obtained (for grade 1 in reading 
and ) for grade 3 in mathematics. No other grade/subject combination 
shows a greater number of positive impacts. The smallest number of 
positive impacts for grade/subject combinations occurred for grades 
7 and 8 in mathematics where no positive impacts were obtained. The 
corresponding mr:\ximum and minimum number of negative impacts are seen 
to be 8 for grade 3 mathematics, and one for grade 7 reading. "59 

Continuing with the TAG summary: 

"An examination of the row totals shows that" (Dallas and) Jacksonville 
(both) exhibited 6 positive impacts. The sites showing no positive 
impacts are Philadelphia, Hartford, (McGomb) , Seattle , and Bronx. The 
maximum number of negative impacts is shown by the (9) negative impacts 
at Seattle. No negative impacts occurred at Athens, Dallas, Anchorage, 
and Jacksonville. The difference between the' number of positive impacts 
and the number of negative impacts is seen to be a maximum at 6 for 
(Dallas and) Jacksonville and a minimum of (-9) for Seattle. "60 



In this appraisal, Battelle may not have clearly portrayed the overall summary 
of results. A more detailed report of these results is given below: 
Heading Mathematics 



GRADE 
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nd 
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nd 
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11 


3 
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8 
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7 
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nd 
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11 
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8 


Reading l8 


■■■63 


23 


7 


1 


16 


1 
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Ik 




M&th 13 


62 


31 


8 


3 


10 


5 


0 


12 


6 


31 


127 


3k 


9 


3 


11 


3 


3 


11 


3 










18 


65 


23 


13 


62 


31 









nd = no significant difference 
+ = significant difference in gain favoring experimentals 
- = significant difference in gain favoring controls • 
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In its findings, Battelle cites 127 instances where there were no sig- 
nificant differences betv/een the performance of experimental and control 
groups. This total represents :-i majority of the 212 possible grade/site/ 
subject combinations. Despite, the fact, Battelle closed its interim report 
rather matter-of-factly : 

'*The lower right-hand totals show that 31 positive impacts 
occurred, 5^ negative impacts occurred and 12? differences 
were not significant. "6l 



To report the documented results, TAG could have used any of these five 
alternatives: 

1. E did no better than C. 

2. E did no worse than C. 
3- E and C did equally well. 
h» E and C did equally poorly. 
5. The results are inconclusive. 



It is also important to distinguish between the two meanings of "significant 
as used by OEO and TAG. For OEO, .5 or better grade equivalent gains were 
significant. Battelle, however, considered gains from 0 to 1.6 grade 
equivalents significant, for example: . 



Grade 




Equivalent 




Numbers 


Number 


0 


1 


.1 


■ 7 


.2 


20 


• 3 


12 


.if 


12 


•5 


9 


.6 


9 


.7 


if 


• 9 


6 


1.0 


1 


1.1 


3 


1.6 


1 


TOTAL - 


85 



OEO and Battelle were talking a different language when referring to signi- 
ficant differences. Lastly, a few comments are in order (l) there was 
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overlap in the MSG-TAC roles and reports, (2) nei-her conducted process eval- 
uation, concentrating instead on program, and (3) Battelle made little 
mention of important E/C differences. 

Final Report From TAG 

Battelle Memorial Institute (BMI) in its final report, elaborated on the 
results and conclusions developed for the OF^ oummary ox Preliminary Results . 
In fact, BMI returned to polish up its work and to shift major emphases. 
Moreover, subtle and obvious changes alike were made. First, low pre-post 
test correlations were no longer listed. Second, the interpretation of 
first grade scores was revised because of the different tests used on a 
pre-post basis. Additionally, the report on the ''incentives only'' sites in 
Stockton and Mesa, the results of retention testing, the comparison with 
spec.al remedial programs were included. BMI qualifies its work in the final 
volume, claiming no knowledge of the tests used for payment purposes only. 

It is useful to compare the spirit of the interim' and final reports: 

'^The hypothesis underlying these two programs was that low-income, 
low-achieving students instructed for one year by regular school 
techniques to w'uich the use of incentives had been added would re- 
gister better achievement on standardized tests than a similar control 
group of students not receiving the incentives. "62 (underlined phrases 
appear only in final report) 

In its interim report, BMI admitted: "Consequently, , for race and income, 
descriptive information (aggregated across sites would be misleading) and 
thus data are given on a site by site basis only."^"^ (parenthesized later 
deleted in final volume) 

As its major concern in revising the interim report, BMI reconsidered the 
characteristics of the participants and their group differences. This 
results in the inclusion of this phrase: 
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'^Thus, of the 1^ sites for which race data were available for 
both experimental and control groups, for 10 of thcoe sioos the 
percentage of whites in the control group is higher than the 
percentage of whites in the experimental group. "6^f 

This leads to the changed conclusions: 

"In summary, often marked differences appeared between 
experimental and control groups within a site in racial 
composition. "65 

Battelle adds another observation, unmade earlier: "The data in Table III 

66 

reveal E/C group differences in income*" (emphasis added) It continues 
by noting differences within sites between elementary and junior high/secon- 
dary levels, which leads to a suspicion of generalizations even on a site 
basis. 

With regard to the education of the participant ! s father, BMI adds: 

"Examination of these data show certain E/C group differences 
within sites. Thus, in 9. of the I8 sites, the control group has • 
a smaller percentage of fathers with * less than high school' 
than does the experimental group, and a higher percentage of 
'more than high school' than does the experimental group. "67 

Further, BMI adds in the final version: "As with the variables of education, 

58 

income and race, E/C group differences within sites occur." To bolster 

its case for E/C group differences, BMI admits": ". . .differences between 

69 

E and C groups in entry level are nonetheless apparent. . ." To document 
this case, BMI cites 8^ of 106 site/grade/sub jec t area combinations at the 
secondary level where "average grade equivalencies are generally higher than 
for the control group. While less marked on the elementary level, the per- 
centage is still high with 66 of IO6 combinations showing control groups 

70 

achievemen*: level entry higher than for experimental." 



In concise narrative and graphic fashion, BMI su-nmariz^^s student progress — 
S und C — in the final report: 
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"As shown in Fin^ur^ 1, both rxperd mental and control students 
in.Cfade 2 r.'r'ii'L o';i at about \he same amount below grade level • • 
and both E ;and C grr ups get further behind grade level with each 
oucceeding vr-ar in school. Also, as the gra^Ie level increases, the 
^.eparation between experimental and control groups increases, and 
more so for reading than for mathematics. For each group in each 
subject area, the rate at which students fall behind grade level 
seems to be about constant from year to year. . .. Finally, both 
E and C students . fall further behind grade level in reading than in 
mathematics ."71 

TABLE IV 

OVERALL MEiVl^ PRE-rE3T GEC VALUES I'OK RE/U)1NG /iND PIaTHEM^.TICS , 
BY GHCuP AND GRI\DE, FOR ALL SITES 
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6A 
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6.6 



An interesting and documented conclusion reached in the final report is 
the following: 

. .the overall entry level achievement of full-year students 
and dropouts is the same, and the dropouts are not a different group 
than full-year students with respect to achievement level."73 

Dropouts and those students who have missed either pre or post tests, amount to one- 
third of the participants in the OEO experiment. The basis for this similarity 
is the baseline data developed by means of a standardized pre-test. 

In addition to considering group differences when interpreting the test results, 
-testing procedures and conditions must be examined. . V\ • .such assessment 
(of classroom test conditions) bear directly on interpretation of the result 

7k 

and conclusions subsequently presented in the analysis section of this report.*' 

With that intention in mind, BMI*s actual description of pre-testing dif- 
ficulties is not altogether persuasive: 
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"In sumiTiary, in all those 15 sites where general or specific incidents 
or conditions were reported, the type of problem reported most often 
centered around discipline problems in the classroom and lack of test- 
baking motivation. To a lesser degree, problems were created by admin- 
istrative confusion resulting from the opening of school, late selection 
of students to be tested, and sometimes lack of communication to 
building personnel. "75 

BMI was selected as testing and analysis contractor for the OEO experiment 
in mid-August, with the result that BMI's role in pre-service training vi^s 
less than desired. In describing its method of data analysis, BMI 
explained alternative methods employed, having deleted mention of "qualita- 
tive judgments" in the interim report in favor of "associated statistical 
tests" in the final version, as a means of obtaining the primary results. 

The interim report was developed by the application of a pre-post model 

to test scores. Group differences on the pre-test are the only ones 

considered quantitatively in the model. The final report went beyond 

the pre-test differences, and considered factors such as race, father *s 

education, family income and parental attitudes. Tn. their own words: 

"These particular variables were chosen because they we^-j judged 
to have a potentially high relationship with post- test perf ormai.'ie , 
because there were observed differences between the E and C groups 
on these variables to provide sufficient data within 
most grade-sites to warrant inclusion in any analysis. "76 

In sum, Battelle * s method of analysis was to "treat the post- test score 

as a dependent variable; treat group membership, pre-test scores, and 

selected other variables as independent variables; and analyze the results 

77 

using regression analysis." 

Defending the rationale of its approach, BMI explains: "Thus, the 

regression technique, in the above-outlines way, "takes into account" cr 

7i 

"adjusts for" initial differences between the groups on the pre-te^t." 
BMI*s revised conclusions follow: 
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'^Thus, the control grcips exhibited statistically superior per- 
forrrance as determined by the regression analyses in approxi- 
r.'-Jbly twice p.s many "nstances as aid the experimental groups. 
However, by far the ma,-) or outcome was no statistically? signifi- 
cant difference between the [;;roips, since 12k differences w^^ro 
not significant. "79 

In unexpected fashion and seemingly contrary to OEO^s purposes BMI adds 
the comment: "It is also interesting to note that 23 of the 28 positive 
impacts occurred at sites associated with three of the six technology 
companies, . . . While refraining from identifying 

them by name in the report, BMI fails to note that all of the three have 
withdrawn from the performance contracting field. 

Using its second method of analysis — the extended variables model — 

BMI reports that: "... application of the EVM provides even less 

evidence in favor of an experimental group impact than the PPM, simply 

8o 

because the proportion of positive impacts is less. 

Contrary to OEO* s' practice, BMI intended the primary emphasis in its data 

analyses to be the results "at a grade/site level." Yet a comparison aggregated 

across sites yields information, of some limited value: "As shown there the 

difference in pre-post gains between the two groups, either in raw scores 

8l 

or grade equivalents, are very small." To sum it up, "In other words, the 

rate of achievement of the experimental groups is not improved to any note- . 

82 

worthy extent over that of the control groups." 

■^•-^sed finally on a more precise analysis and interpretation of preliminary 

results and Oix ^n alternative method f^r analysis of more complex events, 

BMI draws the following, carefully stated conclusion: 

"There is very little evidence that performance incentive contracting, 
as implemented by the technology companies at the I8 school districts 
in this study for a period of one year, had a beneficial effect on the 
readin^^ and mathematics achievement of students participating in the 
experiment, as measured by a standardized achievement. "^3 
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And with respect to the incentives only sites at Stockton and Mesa, BMI 
summarizes: "Overall, there is little or no evidence at Stockton and Mesa 
that the 'Incentive Only 'programs were beneficial to the students in 
rt:^ading or mathematics achievement, as meae;ured by a standardized test." 

The project director sbarted with praise for OEO: "050 had the fortitude 

to take the bold step in sponsoring the project in full realization of its 
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inevitably controversial nature." The directors cited certain constraints 
(found also in the MSG report), centering on the lack of sufficient time, 
but also referring to the lack of early LEA involvement, the. late selection 
of the TAG, and inadequate pre-service training. 

The LSG's were criticized for ill-timed hiring of project administrators 
(LSC representatives) which slowed down the implementation process. They 
concurred with the conclusions reached by the Rand team — that performance 
contracts are developmental in nature, without a set instruc tioral package. 
In their words, "The blend, mix and management of materials making up 

86 

the instructional strategy were ths things that were new to school systems." 

By the end of the year, there emerged a somewhat common core program, which 

witnessed the fact ". . .that each company did not have an individual or 

87 

unique curriculum approach." 

In pinpointing problems, the project directors echoed many of the concerns 
of the MSG — unclear definition of roles between LEA and LSC representatives, 
problems with documentation, curricular areas and their distinctions. As 
regards the MSG, t:ie project directors noted its lack of authority to make 
decisions within tie OEO structure, its defective cost analysis model, its 
questionnable curriculum audit, and insufficient feedback from the voluminous 
documentation. Part of the criticism of the cost analysis and curriculum 
audit was the lack '>f uniformity and substantiation in the collecting data. 
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•In relation to "^.r- TAG, they fault the short prp];^lanning time, ihe wi'^:- 
range in TAG p-:rfc5onn?l, th? late print-out of pre-Lest inform-'^. 1. ion, 
unsatisfactory IPO's ("routine test to stimulate the cash flow for the 
subcontractors'^-.) These criticisms notwithatanding, they had good words 
for improved test conditions on the post- test. 

In their recommendations, the project directors ure*ed improved lin:5s of communica- 
tion, both horizontally and vertically, clearer definition of roles for 
the LEA and LSC, closer attention to local structures and state requirements, 
greater use of legal counsel, use of criterion-referenced tests, better in- 
tegration of the program into the system, ongoing evaluation, penalties for 
LEA contract default or violation. 

The directors press home the need for defining objectives and designing 
appropriate tests to measure the achievement -of t?Lose objectives. Increased 
accountability will be the end result. 

The four LSC's represented in this statement, take serious issue with the 
match (or mismatch) between control and experimental students and the use 
of standardized tests to measure specific learning. In their view, the 
OEO experiment was plagued by (l) limited time for proposals, negotiations, 
familiarization and start-up and (2) an overreaction to the danger of 
"teaching the test." 

Unlike the project directors, the contractors criticize OEO in specifics: 

"To all intents and. purposes the OEO functioned in the per- 
formance contracting project not sis a sponsor but as a research 
institute which delegated only the instructional responsibility 
to the performance contractors. "88 '■•^ 

Moreover, they objected to the "• . .appropriation of absolute authority 

89 

over every aspect of the evaluation process." 
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Standardized tests came under particularly heavy attack. As measures of 
general educational achievement, they are unable to measure basic skills.. 
(Tom Byrns of USOE phrased it well when he said that standardized tests 
did not give a "fine tuning" on performance contracting.) In addition, 
the'LSC^s considered the rate of learning to be as important as the level. 

The four company signatories regarded the OEO project as "as a very large 

quasi-experiment, of limited external validity, fraught with start-up 

difficulties, teacher resistance, poor testing conditions, and other 

■ 90 

problems that adversely affected the experimental groups." ■ As a result 
of this experience, things have polarized the educational comr .unity and 
the private sector. Acknowledging the results of the experiment to be 
"inconclusive," the LSC's urge continued investigation of accountability 
under controlled experimentation. 

\ 
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CHAPTER VI 



LiXAL TMPLirATIONS m) <'0MPLICATI0I-j3 11^3 I-r^RFQRMANOl'] C0NTRACT1::G 



The specter of legal complications has occasionally haunted the performance 



contracting scene. The presence of this check has affected even the boldest 
entrepreneur and given a reassuring sign for the less courageous. Laws • 
have armed the opponents of change, enabling them to have control over events 
through the weapons of injunctions, lawsuits, decommission and 'strikes. 
Whether exercised or not, the threat to use these means has carried real 
clout. Consequently, the bluff has rarely been called. 

In relation to performance contracting, the question — ^*Is it legal?*' — has 
secondary importance. Of greater significance is the matter of what it can do. ■ 
If it can in effect break the poverty cycle -and facilitate learning at less 
expense, it will encounter little opposition. If, on the contrary, it 
flounders and merely aggravates the current dilemma, then the legal process 
will likely confine its existence. 

Charles Blaschke put it rather astutely: *'There really aren't any legal 
problems in education. They're political problems. When a program looks 
like it will be successful and someone's ox is being gored, then a host of 
archaic laws and regulations are discovered." To illustrate the point, he 
cited* the breaking of l86 laws in performance contracting programs, 1? alone 
in Texarkana (based on a thesis by Yale law student Dean Ringel). There are, 
then, means of circumventing, legal barriers when the need for such action 
is compelling enough. 

Legal and Contractual iStipulations 

A fairly substantial amount of ■ material ~ by Martin, Stenner, Adams and 
" Kitchak, Blaschke, Mayhofer and Ringel — has been written on the perforriiance 
contract^proper, with its. basic stipulations. The conditions of the agree- 




ment are largely a product of th'e. circumstances and people involved, with a 
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sia-able domain left to a '^good faith'' basis of action. (This is particu- 
larly true in the area of LEA responsibilities and levels of effort.) In 
view of the short lifetime of performance contracf iiu?;, it is hard to speak 
of the development of an instrument — sa;y, a uniform contract — to establish 
the desired situation. Many of the contracts have been modelled after 
previous ones or prepared by Learning Services Contractor's (LSC's) and then 
abandoned upon completion of the project or turnkeying it. At any rate, it 
is imperative that a study of all of those documents be made to derive the 
greatest advantage from this experience. Certainly there will be a carry- 
over to other general employment contracts. 

The actual number of legal complications that have come about in the 100+ 
performance contracts is relatively small. In fact, the ^practice seemed to 
have enjoyed a kind of diplomatic immunity in its demonstration stages. Where 
difficulties did occur, as for example Gary, there were other extenuating 
circumstances that precipitated the legal action. Where responsible persons were 
by- passed ^ where procedures were not followed, where communication broke 
down, where trust was lacking, - there laws and regulations came in handy.- 
Where forces were divisive, where, goals conflicted, where constraints impinged 
upon individuals, recourse to. legal process became inevitable. 

Writers unequivocally stress the need for implementing a demonstrated program 
according to the "law.'* ■ This especially applies wlien local monies are being 
spent. The underlying assumption is that performance contracts cannot outstrip 
local and state custom without running risks. ' 

Incidently, there may be a certain "fall-out" • factor from performance con- 
tracts. The educational crime of "teaching the test" may not be the exclusive 
practice of the industrial educational complex. Also, parents, administrators 
and even children may not be so easily satisfied when asking for proof about 



the work of a school. With tinie, the waive of perfoririance cc::ti-actir.k-.'- v:ay 
reach even "^he most ordinary School and teacher. 

The Management Support Group/Contractor (.MSG) report on the Office of Economic 
Opportunity (OEO) experiment documents certain legal episodes occasioned by 
performance contracting. In addition, the Rand volumes on Gary and Texarkana 
relate whole series of events in the resolution of conflicts at those two sites. 
With the Dorsett case still in process and a Seattle lawsuit as yet unsettled, 
there is limited information on the details of the charges or the reasons for 
such actions. It is, however, safe to venture that legal action is as possible 
under a performance contract as it is under more common and ordinary contracts 
sales, employment and marriage. 

In at least one case, in the OEO experiment, a ruling by the state's attorney 
general was requested to determine whether a Local Educational Agency (LEA) 
could by-pasii the State Educational Agency (SEA) and even the governor in 
carrying out its contract with a federal aigency. This issue would seem 
to be a power struggle between states' rights and federalism. The performance 
contract may serve only to demonstrate dispute over governmental domains. 
(The OEO contract was upheld in the above example.) 

In another instance, this time in New York, it was decided by either the 
SEA or State Board of Education, that performance contracting was illegal, 
where local and State monies were to be used. When, however, federal dollars 
were available, the obstacle dissolved. Obviously, the issue in conflict was 
economics rather than law, 

A San Diego legal. counsel ruled that performance contracting in California 
was illegal. Shortly thereafter, the California legislature passed its 
^'Guaranteed Learni.ig Achievernent Act of 1971" which established the legitimacy 
of performance contracts in that state. 



A by-product of the performance contracting experience has been the recog- 
nition of- relative naivete of educators in matter of law. With the need for 
increasing awareness of legal parameters, and the accompanying need for" 
legal changes, schools must gear up to achieve any effectiveness in their 
operations when confronted by legal hang-ups. The alternative is for 
schools to become completely defensive and reactive in posture. 

At the same time there is a need for leadership from the schools, who, 
acting out of principle and conviction, bring about legal reform that 
benefits not only schools, but the whole of society as well. Educators, 
moved by conscience and awareness of the public law, can have a powerful 
effect upon generations of people. 

State Legislation 

As noted earlier, recent legislation in the state of California will have 
a decided impact on the future of performance contracting. The law, taking 
effect in March of 1972, has essentially two aims — to improve quality and 
to .cut costs (as is the case elsewhere, the priorities may be reversed). 
Specifically, it focuses on the elementary grades and reading and mathematic 
skills. Passage of this bill approved the use of local, state and federal 
funds for purposes of performance contracts ("Guaranteed learning achieve- 
ment"). Colorado and Florida have passed similar legislation with somewhat 
similar provisions. As is the case in Michigan, participation-is on a 
voluntary basis. 

The thrust of the acts is to make schools responsible for their children/ 
students, not .only to the parents cdid children, but responsible to the SEA 
as well. In the spirit of accountability, schools will be held accountable 
for the achievement of their charges. State control becomes more apparent 
in the requirement for state approval of evaluation designs and exclusive 



control ovv^r r<,£=t iidministration. intended initially as an experimentaJ 
program on a I'-^^pres-r.tative ba^sis, the law remains in effect until 
June j50, 19?5- It assuages teachers- by promising that no certificated .. 
employee shall lose his position as a result. The project is demonstra- 
tive in nature, intended to be a model for replication. 

Jim Mecklenburger regarded the California law as unique in three aspects: 
1) it makes the performance contracting concept legal, 2) the SEA does the 
testing, and 3) it provides for contracting for objectives other than standard- 
ized test scores. 

The law might, however, run into some practical difficulties. Most LSC's 
have left the scene, many for reasons of insolvency. Contractors might 
just finally have the market advantage. One other observation — a reliable 
source has indicated that a California based firm may be the only company 
interested in performance contracts in 1972-73- If that is the case, that 
"LSC is well located. 

In 1971» Illinois attempted to make it legal for school ditricts to enter 
into contractual agreements with private firms or associations. According 
to State Superintendent of Public Instruction Michael Bakalis, the bill, in 
the Senate Education Committee, stands little chance of becoming law. 

Powers and Responsibilities 

For a. closer look at the specifics of the legal matters, it is. well to 
consider the experience of Reed Martin and Peter Briggs who, while with 
"ETS", designed many of the performance contracts including the ones in 
Dallas and Virginia. They are concerned with two specific domains: l) a 
school board's contracting authority and 2) the delegation of power. Per- 
formance contracting is not knd cannot be . an "out". Although advertised 
as. low risk and low cost, it cannot be construed to be a school board* s 



absolving itself of the ultimate responsibility in policy matters. (Gary 
ran into initial difficulties by "turning over" Banneker school to an out- 
side firm to run.. Their contract and experience are instructive on this - 
matter.) 

Similarly, teachers are hired for an express purpose; they cannot be 
circumvented without violating state statutes or teacher union agreements. 
The firm must operate within constraints basically like those under-whlch 
teachers function. Otherwise, independent, profits-making contractors would 
on joy special privilege. 

Martin and Briggs cite testing and payment as the most controversial and 
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most difficult areas of performance contracts.-^ Their provisions usually 

x ■ / 

comprise the major portion of the contract,. Martin offers the suggestion 
that performance contracting,, as a "tool for institutional reform" can work 
toward the end that "schools can increase their? ability to meet their 

97 - ■ 

responsibilities to their state, their students or clients, anii-4h^selves." ; V 



Gene Glass criticizes this form of adverti;s.ed educational accountability: 

"The problem with this ersatz accountability is that it doesn't 
' make the schoolman accountable to the public kt all. It shifts 

the onus of schooling off the schoolman's shoi^lders onto private 
■ contractors, who are all too willing to serve as the whipping boy 

for unsuccessful schooling provided the price is right. "98 

Reed Martin describes a procedure that would keep a school on the safest 

ground: l) setting goals with the public, 2) using a Request For a Proposal 

(RFP) and competitive bidding to select a Learning Services Contractor (LSC), 

3) specifying in the contract the procedures for taking over the operation of 

a successfully demonstrated instructional program.' (turnkey) , and k) providing 

for independent evaluation or audit procedures, to maintain credibility in 
99 r 

the public's eyes. (Notwithstanding the logic of his argument, it should 

O 

ERiC ^® pointed out that Martin was employed by "ETS" , a firm providing 
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technical assistance to schools and featuring MSG, evaluation and audit 
capabilii;ies« 

Roald Campbell and James Lord on in their book, devote one section to a 

discussion of legal implications. They refer specifically to the Office 

of Economic Opportunity (OEO) contracts and the matter of a school board's 

"supervision and control." They suggest a natural inconsistency in the 

role of the LSC, who is either an independent contractor or an employee of 

the school board. If he is the former, then the board has relinquished 

its control. If he is the latter, t-;en the board is liable for his 

program, which would have major implications. The authors, however, speculate: 

"Courts in the past have not let the rigid rules of hornbook law 
hamstring programs whose underlying policies were sound, and there 
is probably enough flexibility in the common law to allow it to 
reconcile two clauses/ incompatible at first glance •"100 

The OEO experiment introduced a "federal presence: into performance contracting 
(sometimes affectionately known as the Washington syndrome). (OEO for example, 
in contracting primarly with school boards for remedial education, including 
contractual provision for "non-discrimination" and "equal opportunity" 
clauses.) It also succeeded in enacting certain promises from LEA's: 
to supply information, to give "notice prior to publication," to develop 
community sentiment, and to assume the burden of recovering overpayments to 
the subcontractors — LSC's« Moreove.?, GEO reserved the "right to inspect" 
all work done by the LSC at "all reasonable times." (There would probably 
be bases to test the legality of this "intervention.) Finally, LSC's were 
required to post performance bonds, were assumed to be in a direct relation- 
ship to OEO, a relationship that theoretically superseded the authority of 
the SEA and even the governor . (ThiF Wv^s contested in one state, but .the 
interpretation was upheld. )"^^"^ 

The OEO-Testing Analysis Contractor (TAC) contract also reflected the 
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^'federal presen'--.'* Of special interest was the explicit provision of the 
names of the' people considered essential to the evaluation program, Battelle 
Memorial Institute - Columbus, Ohio (BMT) disavowed any conflict of interest 
or any financial intereot in the LSG*s; it agreed to the "notice prior to 
publication^^ clause; and promised to stay out of the field during the course 
of the experiment (i.e. to remain disengaged from performance contracts). The 
provisions were in accordance with OEO^s"rigorous evaluation design," whic^ 

program success or failure turned on objective and disinterested testing 

. ^. 102, . 

and evaluation. 

In summp* y of their discussion, Campbell i-.nd Lorion predict: 

"So long as the purposes of the new practice appear to be 
desirable, it is unlikely that the courts will impede the 
growth of the field, and performance contractors and school 
officials usually will be left on their own to resolve the 
educational and administrative disputes that will inevitably ' 
arise ."103 

In Summary 

A few final comments on- the legal implications of performance contract^Lng. 
First, federal interpretations have playid a not insignificant role." For 
example, Texarkana started with Title VIII ESEA and Model Cities funds. 
Since that time other monies have been made available — namely ESEA I 8c III, 
which has facilitated the development of performance contracting to this 
point. When Title I monies were declared eligible for use in performance con- 
tracts, it became possible to plan som&thing like the Virginia state-wide 
projects. Other states, however, have not followed suit, reserving the* right 
to make their own funding decisions. Consequently, performance contracting 
with Title I monies has been virtually impossible in ""tain states.. 

A minor problem has arisen in conjunction iwith the materials developed over 
the course of the^ program. In some cases, LSCs have been able to copyright 
the materials produced by research and development in public schools. 
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Finally, a sign of progress is the occasional provision for student rights, 
which are recognized by the contractor and all of his personnel. For too 
long, schools have not been learner-centered. By recognizing a basic bill 
of rights for students, a new attitude is manifested. This should check 
traditional manipulation of students for purposes of experimentation. 
It should also challenge those who speak of low risk, low cost programs. 
If students are the most important, then their welfare should assume top 
priority. 

At an Educational Technology Conference and in a paper prepared for the 
American Educational Research Association, Myron Lieberman discussed the 
matter of employment contracts and the effects of performance contracting. 
He -distilled a number of issues to which the American Federation of Teachers 
(AFT) had initially responded in gut-level fashion. For example, he 
brought up the matter of a school's responsibility to its teachers, new 
and altered job descriptions for teachers, and teachers' ' share in increased 
productivity (cost-effectiveness). 

Lieberman contended that in contracting out "work'' a school. acts counter 
to collective bargaining laws. It also runs the risk of delegating " 
powers specifically atssigned to a school board. OEO came under criticism 
for not considering the questions: What do we do if it works? What are 
the implications of a cost-effective instructional program? Can it be per- 
. mitted to^ deny "working agreements developed over time? These and many more 
implications must be addressed if performance contracting is not to offend 
the entire educational community. 

As an innovative program, performance contracting- has served the function 
of a Trojan horse, transporting new and used methods and materials which 
are accidental, not essential characteristics of performance contracts. 
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These fellow-travelery have included: research and' development incentives/ 
contingency management, differentiated staff ing, tea-hing machines, t-eachin- 
"instruments", desegregation, standarized tests, evaluation, audit, in-servicf/ 
pre-service training, management support, criterion-referenced tests, merib 
pay and programmed learning. The movement resembles closely the legislative 
process, whereby riders are attached to bills r .'ten indiscriminately. 
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CHAPTER VII 

ROLE OF THE STATE EDUCATIONAL AGENCY (SEA) - TRENDS AND DEVELOPMENTS 

C 

^ _ There _are important lessons in the performance contracting experience for 
state educational agencies (SEA's). The answers to a questionnaire sent 
to all Chief State School Officers made it appear that state educational 
agencies (SEA*s) have not (."ompiled a compendium of information on this 
subject. While the responses were reasonably prompt, and the return was 
good C^8 of 50), the> information was not extensive and often quite incom- 
plete. 

For an SEA to follow the, scenes closely, it needs the staff flexibility 
to take on assignments as they develop. It also requires a research and 
development capability to conduct in-depth study of contemporary issues 
such as performance contracting;. It needs a sure contact with the field 
in order to feel what is going on in our schools. 

SEA Postures on Performance Contracting 

State educational agency (SEA) responses to programs such as performance 
contracting can assume a variety of shapes. The Florida SEA, for example, 
has taken a neutral position in order not to interfere with local educa- 
tional agency (LEA) efforts. Colorado, Michigan, Virginia and California, 
on the other hand, have launched state-wide projects for purposes of demon- 
stratioa and replication of measures of accountability. 

Hawaii developed a position pap^r for the state board of education. At the 
same tiri.e, it undertook a survey of mainland performance contracts, expecting 
to report the results in June, 1972. In its actions, the Hawaii SEA played 
a research and developmen^': role for the state board of education. 

Other SEA^s approach these matters on a regional basic, cooperating to 
develop extensive reports of timely relevance to participati.ng states. 
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•For exan^ple, Upper Mid-West Region for Tut'^r^state Projects (UM^^TF) sponsored 
a previous conference/study and supports this continua i^ion project. Farther 
west, the Rocky Mountain Region pi-epared a guide on performance contracting 
also. While the regional efforts were underway, statf projects were released 
which had been undertaken separately-e.g. the Michigan guidelines and the 
New Mexico reports on performance contracting and on accountability. 

State Involvement in Performance Contracting Activities 

Obviously SEA*s have different levels of involvement in matters like per- 
formance contracting (which has been tested in some 51 states). There will 
then be different degrees of responsibility for. state projects, depending 
on the funding source. 'Indiana, New York and Texas were forced to react to^ 
specific developments within the state. The Indiana SEA, embroiled in the 
Banneker controversy, supported state standards for teacher certification 
and for instructional time allotment; the result was a confrontation, re- 
solved partly through intervention of John L- Loughlin, Superintendent of 
Public Instruction, and the Indiana SEA staff. 

In Texas the state attorney general was asked to rule on the legality o:* 
educational performance contracting on a demonstration-i-,e. with the inten- 
tion of turning the operation back to the school upon its successful demon- 
stration. The method was determined to be legal. 

In New York, for various reasons it was decided that performance contracting 
was illegal - an interesting way of saying that federal money cojld support 
such programs, but not local and state dollars. 

What then are the alternatives for SEA*s when keeping an eye on developments 
in schools within the state and beyond its borders? There is no pat answer, 
^t depends largely on SEA resources. Probably the hardest and most time- 



consuming task is gathering accurate, first-hand information. How that infor- 
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mation can then be made a^'ailahle to decision-makers is a dissemination pro- 
blem. 

Conferences are seasonal, somewhat expensive and of varying effectiveness. 
A source of surprise from the returned questionnaires was that only 3 states 
reported holding conferences or workshops on performance contracting (Minne- 
sota, Missouri and Iowa). 

A useful device in this whole area would seem to be the simulation exercise- 
e.g. the simulation of negotiating and planning a performance contract as 
developed by Mecklenburger and Saretsky for the 1972 AERA conference. This 
i^erious kind of role playing and interaction leads to a '^'^'^ling and appre- 
ciation of the interests and people involved. 

Films are inexpensive, although they quickly become dated. The Performance 
Contracting Experience in Grand Rapids , directed by Mecklenburger with help 
from V/ebster and Indiana Un .versity, A-V Department, gave a real sense of 
the instructional program and how it worked on an individual level. Visual 
description has no peer, especially in depicting people-processes ^ (Reading, 
'Ritin * , 8; 'Rithmetic, Inc., produced for the television show Black Journal , 
was a one-hour program devoted to the Banneker performance contract in Gary, 
Indiana. In introducing the program and its personnel, the film was rich in 
personal detail), SEA's have not begun to touch the limits to visual techno- 
logy. 

SEA's might provide legal assistance to LEA'S, who are physically or finan- 
cially limitec i.n that rer^urce. This would seem to complement other state 
supervisory responsibilities. 

Finally^ SBA's, singly or in combination with public insbitutions of learning 
^ could develop capabilities for rendering technical assistance, evaluation or 
ERIC audit functions within the coordinated system of education. This may be a 
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means to imyrove and to restore public education. A sales pitch won't work 
when the customer wants proof. 



CHAPTER VIII 



IN CONCLUSION 



Where does it all lead? What will become of performance contracting? 
Here are excerpts from, performance contractors^ judgments. You decide 
what they mean: (emphasis adc d) 

"Performance contracting is a way to know of our other services 
related to accountability in education. ... We are currently 
working in conjunction with one of the world's leading accounting 
firms to plan and assist in the implementation of state-^wide and 
major districts' accountability programs :*n a PPBS formav,." 

''In addition to contracting for elementary .^nd secondary education, 
.v*e have a new offering in the form of in-service staff traini ^ng 
programs for local school districts in human awareness. " 

"... we do not intend to use performance contracting as a maior 
marketir.p^ tool, rather we are using it as^a demonstration technique 
of the offectivenss of our systems in certain areas. We feel 
strongly that a performance contract should not be used as a 
vehicle to test absolutely new materials, as t: -s puts all the 
risk on the school systems and none on the educational materials 
producer." 

"We have purposely limited our activities in this area (performance 
contracting) beacuse of the limitations of available measuring instru- 
ments as well as the relative unsophisti cation of many school districts*" 

"The Board of Directors of x has decided not to pursue additional 
activities in the jerformance contracting area because of the financial 
exposure required, based upon the use "of norm referenced tests to 
measure student achievement . . . it is rather clear that norm refer- 
enced tests are totally inappropriate as measuring instruments for 
this purpose. ... I believe the concept of accountability is worth 
saving.'* 

"9 . .we have been expanding our capabilities, planning and developing 
a variety of managerial innovations for SEA's , which we feel will be- 
corns increasingly more responsible for 'honing the current edge' of 
educational reform. 

. • .we are studying distribution formulas which take into account 
the cost variances and incentive.v; required to encourage efficient 
management at the LEA leve l." 

- "The OEO projects were set up to fail, not GEO... There was a hidden 
agenda ... it was an OEO failure from the start. Companies never 
had a prayer." 

"When subject to the same general constraints, a private company 

can't do any better than a well-administered school • . . We shouldn't y 

deceive ourselves in saying we're using the best instructional methodology 

now. . .we. are still ready to take on a performance contract, anytime, 
anywhere." ^ 




"V/e wore able to got the^se truly dramatic .r;ain.- because, of siir;- 
f-'Tstioris and input fx*om administrators and teachers in Gchooi;;: 
like yours located all over the country. 

Perhaps, you have a problem in your school sy^t^ms ( SEA ) with whicii 
-'C could help you . We are very r)roud of our [rogi'c'vns and would be 
uipipy to work with you." 

"This guarantee, the first such plan offered by a laajor publisher, 
is one approach in the general area of accountability « It is not 
— performance contracting in the usual sense of the word. (We) sell 
(our) program to the school and provide consultant services . All 
aspects of the use of the program are under the control of the s chool 
and its personnel. Tae school even selects the pre- and post-test 
ins^^truments." 



It is also important to consider the words of those who were actively 
involved in actual performance contracting progranis. From tneir impres- 
sions plus those of the contractors a prediction can be made: 
"Nobody seems to like it but the customers that buy it," 

"It offers an 'economically feasible way to pilot new instruc- 
tional systems." 

"If performance contracting is allowed to stand or fall on its 
■ instructional and management merits and if it can be- evaluated 
objectively within feasible expectancy levels, it might provide 
education a powerful management tool ." 

"Little is known about performance contracting. Part of the 
reason is political, since there are empires to protect. » • 
Performance contracting has disappeared or will disappear be- 
cause of testing ." 

"You can^t continue to fund unless you have federal funds ," 

'^Dramatic and quick results aren't realistic . Development of the 
^, Salk \jaccine came about because of the trial and error method." 

"It is likely that this coming year school personnel and communities 
will press for this, type of institutional flexibility . Schools and 
teachers would then be able to be much more effective consumers of 
what private industry and others might have to offer through perfor- . 
mance contracting." 

" X is likely to be the only company to make money on. performance 
contracting." 

// 

"Performance contracting trotted a skeLe .ton in the closet out into 
the open." 

"The ultimate goal is individualized instruction, which is the right 
Q materials to the right kids at the right time." 

sic 

"We hope next to be able to say very positively that this is what we c 
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do and this is what we cajino'c do in education. This process should 
giv€ us the data necessary to ask for money that we absolutel' nust 
have to do the job and to say *'no^' to funds that exceed those ae- 
maiids.'^ 

"... it may not be too long before teachers sign 'incentive' or 
' bonus * contracts under which they'll share money that is saved 
through more effective teaching." 

"Beacuse of its uniqueness, bilingual education would be difficult 
to be considered under any performance contract." 

"Performance contracting didn't do any better, but it didn't do any 
worse ." 

". . .somewhat incautiously arid prematurely , we invited the public 
to witness the educational miracles technologies are capable of — 
and then • • . failed to prqdi-.ce anything like a miracle or even 
a respectable demonstration of the potential of scientific device., 
and techniques, in the direct service of -the learning process." 

"The purpose of performance contracting is to ' infuse ' potentially 
promising programs for- purposes of strengthening education .'^ 

"The first generation of performance contracting hat reached a plateau ." 

"Accountability has become a buzzword . . .. Early returns on performance 
• contracting were weighed too heavily. It was a first reading ." 

"Unless schools, as cor'.onmers of education suppliers, insist on account- 
ability from business, there is apparently no magic that would guarantee 
greater responsiveness under a performance contract." 

"Monies are now being spent on cuvricular materials selected 'to the 
kid. . . This is a better way of me ensuring what schools are doing ... 
The greatest " support comes from the public." 

"If you can't ride a bike, why put a motor on it?" 

(In conclusion) To draw together the preliminary judgments already made in 
this paper, it seems appropriate to describe educational performance con- 
tracting as a frontal wave, which has crashed against the rocks of educa- 
tional institutions and traditions. The roar was deafening, the spray 
wide-reaching, the style majestic. For a moment, the wave covered the 
scene. But the rocks stood firm and the sand filled in the tracks. 



As a single wave, the energy of performance contxacting is spent. As part 
of a larger change process, however, even smaller, even further apart waves 




follow relentlessly in its path. Meanwhile, another frontal wave is in the 
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making for education will not, cannot remain static. 

The rocks iiave taken on a glow from the washing hy the wave, hut the end 
result is a shaping of the shore iii forceful yet artistic manner. The 
weathering process in American education is not to be denied. 
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APPENDIX 



As one climax to this study of performance contracting, a conference was 
conducted in May, 1972, in Bloomington, Minnesota. This conference was 
sponsored by the Upper Midwest Region for Interstate Projects and featured 
presentations by ma>">y nationally recognized experts cn performan.ee contracting. 
Papers c.elivered by five of these presenters are contained in this appendix. 

Specia] thanks is due to each of these noted commentators and to such other 
persons as Dr. John W. Porter, Dr. Joan Webster, and Dr. Brian Fitch for 
contributing to the success of the conference. 
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A bLi::-i\iiv or tie iiA'^'i/niiiv s'iudy of i:ix!c:vrioNAL 



CiBORC.r: R. IIUL 
RM'.^ iJorpurat ion 
May 0, 19/2 

imHODUCTION 

As this group is well aware, for about three years business firms and scliool 
districts have been enter±iig a controveisial relationship called performance con- 
tracting* The commercial firms provide instruction to public school students and 
their pay is a functior, at least in part, of the achievement gains of the students. 

Achievement gain or cognitive growth is usually measured by standardized norm- 
referenced tests sucli as tlie Stanford Achievement Test, Iowa Test of Basic Skills 
and like instruments. At the start of a performance contracting program a student is 
given one of these standardized tests: At the end of the program he is given another 
form of the same test. The pre- and post-test scores are subtracted, the difference 
is called the gain and is used to determine the contractors^ fee. Some programs 
have experimented with other types of achievement measures. Criterion referenced 
tests, for example, have been used in several programs. Adequate altematiyes to 
Standardized norm- referenced tests have yet to be found, however, and all programs 
utilize such tests at least partially and often exclusively. 

A performance contractor comes into a school and sets up. an instruction program, 
usually to teach reading, sometimes mathematics, and in a few cases vocational subjects. 
He uses new materials and tediniques and sometimes new equipment. There often is 
enphasis on individualization - diagnosing each students' weaknesses and strengths 
and providing materials tailored for him. 



* The views and conclusions contained in this talk are those of the author and should 
not be interpreted as. representing the official opinion or policy of RAND or the 
Department of Health, Education and Welfare, 
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Frequently the classroom environment is clianged. Often carrels with cassette 
tape recorders replace desks. The teacher usually, but not always, remains on the 
school payroll. lie often is assisted by a paraprofessional. The teacher operates 
more as a diagnostician and manager and less as a conveyor of instruction, compared 
to a teacher in a conventional classroom. 

Local school officials have expressed the need for materials to assist them in 

deciding about performance contracting programs. In June 1970 the U.S. Department of 

Health, Education and Welfare decided to sponsor the preparation of a guide and re- 

99 

quested RAND to undertake the project. In addition to the guide, we have published 

100 

a theoretical analysis of this .^arrangement , and a six-volume report detailing and 
analyzing our field studies of a sample of performance contracting programs?''" All 
of this work has been performed pursuant to Contract HEW-OS-70-156. ffy remarks today 
will summarize the major conclusions stemming from RAND's field investigation under 
that contract. 

RAND examined about 20 programs. Eight programs in five cities were investigated 
in detail. These eight programs are shown in this Table. 

Performance contracting began with programs in t\^ro cities in 1969-70. During 
the 1970-71 school year the Office of Economic Opportunity sponsored a 20 project 
structured demonstration, and at least 50 or 60 other programs were funded from various 
sources . 

The OEO demonstration is a good example of the large social interventions or 
Quasi -Experiments that are more and more being used in educational research. Most 
non-OEO programs do not exhibit as much an experimental control or design but they 
may be more typical of future programs. 

The sudden popularity of performance contracting stems, I believe, from hopes that 
it might address one or more of three current educational policy concerns: 
° How to improve the achievement results in compensatory or 
remedial education programs. 
Q ° How to develop educational accountability systems - that is, 
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accountability for results as well as inputs, 
° How to overcome tlie barriers to teclinological innovation in 
the public schools. 

Let us briefly examine the relevance of performance contracting for these three 
concerns . 

EFFECTS ON ACHIEVEMENT , 

The premier hope for performance contracting was that it might end the gap 
between achievement test scores of students from privileged backgrounds and those 
from homes with less advantages. 

This Table shows average achievement gains for the eight programs in the RAND 
sample. For exajnple, in the first grade in Gary the students gained, on average, 
1,7 achievement years per year in reading and 1.7 achievement years in mathematics. In 
the other grades the achievement was, on average, 0.7 of a year in reading for the 
school year and 1.2 achievement years in mathematics. 

Grade 1 results are shown separately since it is difficult to know how to 
interpret gain scores for first graders. The results ranged from: (1) somewhat 
better than comparable groups of students, as in Gilroy, for example, to (2) no 
better than like groups, as in the Norfolk seventh grade*, for example, to (3] worse than 
like groups, Texarkana and Norfolk fifth grade. Recall that most programs did not have 
formal controls so these judgments are based on selecting groups that seemed like 
the treatment groups. 

j To sum up, we found no substantial or even any consistent advantage for performance 

I contracting over other types of instruction, I think that this conclusion from the 
RAND sample is supported by the results of the OEO program. For purposes of compari- 
son with the other achievement figures you might be interested in the OEO results 
summarized in this Table. 
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Table 3 

OEO PEiy^ORMANCE CONTRACTING EXPERIMENT 
Aggregate Mean Gains 
(In years of achievement gain) 



Grade 



Experimental 
Gain 



Control 
Gain 



Difference 



NA 
.4 
.3 



NA 
.5 
.2 



+ .1 
-.1 
+ .1 



.4 
.9 



.3 

1.0 
.8 



+.1 
-.1 



Math 

1 NA- • NA . --- 

2 .5 • .5 

3 .4 . .4 

7 .6 .6 

8 .8 1.0 -.2 

9 .8 .8 



SOURCE: Office of Economic Opportunity, M Experiment in Performance Contracting ; 
Summary of Preliminary Results, OEO, February 1972, pp. 35-36. . 
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There are three possible explanations why more substantia] gains were not achieved 
last year. The first is that the systems that the contractors had were simply no 
better than those already used in the schools. The second is that the 1970-71 
results may reflect the frictions and start-up costs involved in the first year 
of any innovation. These programs all turned out to involve major developmental 
efforts and the results may reflect these efforts. Third, the tests and measures 
used may not be adequate to measure improved teaching effectiveness. I will say more 
about this possibility in a minute. 

The point I wish to make here is that the data in our study or the other studies 
I have seen are not adequate to make a scientific judgment between these three possible 
explanations for the 1970-71 achievement results. 

The costs of these programs seem to be a bit better than the usual remedial 
program. We computed the instructional costs - not the total costs - for various 
programs on a comparable cost basis* We standardized factor costs and assumed 
operation of the program by the school systems themselves. On this basis we estimated 
that a performance contracting program in the RAND sample would cost in the neighbor- 
hood of $125 to $200 per student per subject. A conventional remedial program, esti- , 
mated in this same fa.shion, would cost around $200-$300 because -such programs tend to 
be more labor-intensive. Performance contracting programs tend to substitute para- 
professional labor and materials for certified teachers, compared to the usual 
remedial education programs. 

ACCOUNTABILITY 

In addressing the problem of attaining educational accountability, the 0.1 re- 
sult in the fifth grade at Norfolk provides a helpful illustration. It implies that 
no learning took place. The contractor had promised Norfolk an average of 1.7 years 
learning gain per student. When he gave the pre- tests for the fifth grade, however, 
he fouffid that the students were functional illiterates. He started to teach the 
"tudents work-attack skills - how to listen to sounds and recognize them on the 
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printed page. Each student was assigned learning objectives. The independent evalua- 
tor from the University of Virginia, using a learning mastery test, tested tlie students 
on the achievement of the contractors^ objectives. Most styidents mastered 80 to 90 
percent of the objectives, according to this test. 

For contract payment purposes, however, the students were administered the voca- 
bulary and comprehension sections of a standardized norm- referenced test. They 
showed no gain. The State of Virginia, the program sponsor, concluded that the 
contractor failed because the fifth grade students' reading skills didn't improve. 
The contractor claimed the low score was because the norm-referenced test didn't 
reflect the content of the program. He cited the high scores on the learning 
mastery tests as evidence he had been successful. The State of Virginia suggested 
that perhaps the students had mastered the objectives before the program started. 

This dispute has an important implication for accountability. The standardized 
tests that are used to evaluate these and other innovative programs were not designed 
to measure output or instructional effectiveness. They were designed to predict aca- 
demic success so students could be classified and assigned. In using these tests for 
accountability purposes we run into severe statistical problems. More important, we 
run into questions of whether the items tested on the standardized norm- referenced 
tests match what we want to have included in our curricula. On the other hand, 
criterion- referenced tests have not been sufficiently standardized to provide the 
objective measures that school districts desire. Criterion- referenced tests not only 
are difficult to interpret when used as a performance measure, but they create serious 
logistics problems. The OEO programs, as well as those in Virginia and Texarkana, 
encountered serious difficulties in trying to implement criterion-referenced testing 
programs . 

In short, I interpret the performance contracting experience to indicate that 
considerable development . work is required before we will have tests and measurements 
adequate t6 achieve the policy goal of educational accountability. 
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TECHNOLOGICAL CHANGE 

Let us consider the third point, the contribution of performance contracting to 
educational change. This appears to be where perfoiTnance contracting may make the 
most important contribution. In the programs we have observed, even those where the 
achievement results have been unspectacul?*:, the new materials and techniques have 
created interest and seem to be leading to some further applications. The introduc- 
tion of outside programs may be a catalyst for getting more individualized instruction 
into the schools. Outsiders, it appears, in some cases can better overcome inertia 
and attract more attention than insiders working within the established rules. 

I should say a word about the contractors. Westinghouse Learning Corporation, 
which had two of the programs we studied and three OEO programs, and CMES, the contractor 
for another project in the RAND sample, have gone out of the performance contracting 
business. Several firms involved in the OEO experiment have also left the industry. 
Performance contracting by itself has not been lucrative. It has, however, led to 
follow-on business in some cases. Also, it has enabled a number of new firms to 
break into the educational materials and services market that has been dominated by 
old- line textbook publishers. Some school districts are willing to try out new firms 
if they will give performance guarantees. This is one reason why performance con- 
tracting may be around for a while. 

THE FUTURE OF PERFORMANCE CONTRACTING 

The 1970-71 erperience dashed the hopes of those, who believed that performance 
contracting would be an easy and dramatic solution to Americans compensatory education 
problem. It is also clear that problems of obtaining valid and reliable measures of 
instructional success remain. Thus, it is unlikely that performance contracting will 
regain its past popularit)-. 

Performance contracting, however, might play a limited educational role. As a 
means to facilitate curriculum innovation and as a way for new firms to share some of 
the risks involved in new materials or procedures it has some attractive features. 
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Moreover, some of the learning systems used by performance contractors may have modest 
cost advantages relative to conventional approaches to remedial education. If so, 
even if achievement gains do not increase substantially, these modest cost advantages 
might be converted into modest improvements in instructional cost -effectiveness. Such 
conclusions must be stated in tentative terms since all the 1970-71 programs involved 
extensive start-up friction and costs as well as major development efforts and so it 
is difficult to extrapolate to some future * 'steady- state** period. 

In short, performance contracting is no panacea for Americans educational 
problems in general and its compensatory education problem in particular. It may, 
however, be a technique that can make a modest contribution to education. 



.^V EXAMINATION OF EVALUATION DESIGNS 
FOR PERFORMANCE CONTRACTING 



By: 



STEPMEN P. KLEIN 
Educational Evaluation Associates 
and 

Center for the Study of Evaluation 



**Does educational performance contracting work?*' Tliis question, and this one only, 

was the focus of over a million dollars worth- of research funded by the Office of 
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Economic Opportunity. OEO examined the results of their study and reached the 
conclusion: "No, it doesn't work/' Now, if you believe that the OEO study was a fair 
test of this question, and accept their conclusion, then you would also have to agree 
that it would be a waste of time to hear me talk about evaluation designs for per- 
formance contracting. If performance contracting does not work, as OEO claims, what 
is the point of talking about it further? 

I see by your continued presence that there are at least a few doubters. Perhaps 
you feel as I do, that asking the question: ''Does performance contracting work?" is', 
just like asking the question: "Do books work?" The answer may lie in which books, 
under what conditions, with what kinds of students, and for what purposes. It is 
conceivable that one might get quite different results from two kinds of books, just 
as one might get different results from two contractors. 

For these reasons and others I will present, the issue is still open as to 
whether performance contracting is an effective technique. So, we can turn our atten- 
tion to what kinds of designs would be most appropriate for evaluating a given per- 
foxmance contract. Tlie specification of these designs is a function of several con- 
siderations. The first of these is "For what kinds of decisions about the performance 
contract do I need to gather information?" Do I just want to find out how much to pay 
the contractor, or are there other kinds of decisions for which I need some data? For 
example, would it be economical to continue the contract for a few more years relative 
the levels of student performance being qbtained?. Could the school take over the 
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contractors' instructional procedures with equal or greater efficiency and effective- 
ness? Are the side effects accruing from having the contract of sufficient import as 
to influence the contract's total effectiveness? Some o£ these side effects might be 
renewed vigor on the part of the regular teachers to keep up with the contractors or 
too much attention to skills assessed by the payment test and not enough to objectives 

like improved creativity and self -concept. In one study my firm did for a California 
103 

school district, we asked teachers to cite examples or indicants illustrating how 
their teaching effectiveness iinproved during the past year. We then analyzed the 
answers for two groups of teachers. The first group were teachers whose principals 
used the traditional teacher evaluation system of making observations and ratings, 
while the second group had performance contracts with their principals regarding stu- 
dent achievement. The results were quite conclusive and indicated that teachers who 
were under the old evaluation systona reported how they inproved in terms of their 
teaching techniques while the teachers under the performance contracting system cited 
evidence of their improvement in terms of student achievement . I am sure you will 
agree that this type of attitude change on the part of teachers as a consequence of 
performance contracting is likely to have a significant impact on their student's 
achievement. The point here is that the evaluation design for a performance contract 
should consider the major kinds of decisions that have to be made and the kinds of 
data needed for these decisions. 

The second major consideration that influences the evaluation design for a per- 
formance contract is 'TVhat is the purpose of the contract?" Is it to improve student 
skills in certain areas? Is it to prevent dropouts? The answers to these types of 
questions dictate the kinds of measures that should be used in assessing the de^Tee 
to which the program was successful in achieving its objectives. Many school dis- 
tricts side-step this issue by just specifying a given test as the criterion of success. 
This approach fails to recognize the fact that one measure of a general sl<ill, such 
as reading, may assess many different objectives than another measure of this same 
global skill. Even when objectives are comparable across measures, the enphases 
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placed on them may be quite different. It is quite possible that focusing instruction 
on a certain specific skills such as letter recognition, might improve performance 
dramatically on one reading test and not on another, simply because one of them has 
many inore items measuring this ability. Thus, it is wise to specify the kinds of skills 
and concepts you want the pupils to achieve and their relative importance before you 
select or construct measures to be used in the evaluation. Only in this way can you 
be sure that the changes you obtain in student performance on a test are the kinds of 
changes in which you are interested and not ones that are just tangential to your 
major areas of concern. 

A third consideration that influences the evaluation design for a performance 
contract is 'TVhat procedures did the contractor use in achieving the program's 
objectives?'' This information is needed to find out whether the program was really 
put into operation as planned so that if the school decided to take it over in the 
future, they would know what to do. For example, a program might achieve its goal of 
significantly reducing dropouts simply because they employed good-looking instructors 
rather than any fancy electronic equipment. As we all learned from the Texarkana 
fiasco, it is imp(|)rtant that the evaluation design take into consideration the possi- 
bility that the contractor's procedures might just be teaching the specific answers 
to the items on the payment test. One way to control this problem is to make 
periodic unannounced checks on the curriculimi materials and instruction being em- 
ployed. A second method is to use an item pool for constructing the payment test that 
is large enough to make impractical any attempt to teach all the answers. The use of 
periodic checks is more appropriate for a single school district, while the large item 
pool approach is more efficient if many districts are involved in the evaluation. 

Now let us turn to the fourth major issue influencing the evaluation design for 
performance contracting, namely, ^TVhat measures should be used?'' There are two basic 
approaches one can take: selecting existing measures that provide the best overlap 
with the particular objectives you want to achieve or constructing' measures for these 
|gj^0ectives. The advantage of the first approach is that existing measures have 
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generally gone through rigorous development and field testing. The major problem with 
such measures is that they often do not overlap well the particular objectives of 
the program. Tliis means that they tend to be relatively insensitive to actual changes 
in desired student performance. Given the history of educational programs and the 
general insensitivity of these measures to program objectives, it is highly unlikely 
that such instruments would show any major changes in student ability in just one 
year's time. Often, this problem is compounded further by the exceedingly poor per- 
formance and dismal prognosis of the students in the program. Thus, an expectation of 
one and one-half year's grade equivalent gain in test performance in one academic 
year is like expecting a mouse to get an elephant pregnant. In the Office of Economic 
Opportunity's case, the mouse's claims to prowess were so convincing that OEO became 
a believer. A more reasonable goal for use with existing instruments would be one 
year's growth in grade equivalent for one year in the program and this result would 
be likely to occur only after the program had had a chance to run for several years. 

The second approach, constructing rather than selecting measures for particular 
objectives, has the relative advantage that the measures produced are potentially 
more sensitive to the particular objectives being taught. However, quality test con- 
struction is not simple - one does not produce valid instruments just by fiat. It 
is possible to construct good measures of particular objectives; it just takes a lot 
more time, money, and expertise than is often devoted to this task. Defining the kinds 
of items that should be constructed to measure a given objective also is a. job in 
itself. Robert Stake recently illustrated this point by listing the various kinds 

of items that might be used to measure a single piece of knowledge, namely that Point 
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Barrow is the northernmost town in Alaska. The items he cited that could be used 
to measure this knowledge were as follows: 

1. What is the northernmost town in Alaska? 

2. What distinction does Pt. Barrow have among Alaskan villages? 

3. The dots on the adjacent map represent Alaskan cities and towns. One 
represents Pt. Barrow. Which one? 

4. What "would be unusual about summer sunsets in Pt. Barrow, Alaska? 
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It is evident frpm this list that different levels of ability and understanding are 
needed to answer these questions. The practical implication of this fact is that the 
school district and the contractor must agree on the particular kinds of items that 
will be used in evaluating the program - and let me warn you now, this is by no means 
an easy task. 

Thus, neither existing instruments nor specially constructed ones appear to offer 
a siji5)le solution to the question of just what measures should be used in evaluating 
student performance. One new development in the field of educational measurejnent 
may at least offer some help in resolving this dilemma. This development involves 
banks of objectives and items to measure them. Several of these systems are being 
field tested now in various parts of the country and I expect that before the year is 
out we shall see a few of them ready for general use. 

Even if we are able to resolve the problem of defining what measures we should 
use in evaluating performance contracts, we will still have to ask the questions: *'How 
do we analyze the results*'? and "How do we pay the contractors?** In answering these 
questions, let us first look at some errors common to evaluation designs for perfor- 
' mance contracts. Briefly, the first of these mistakes is as follows: Some studies 
start off well by using two groups, experimental and control; but in selecting stu- 
dents for these groups, there seems to be a tendency to try and put the students with 
the lowest ability into the experimental group. Miy contractors do this, rather than 
balance out ability levels between groups, is unclear; but in so doing, they open a 
pandora's box of problems and biases. These biases and problems are so great as to 
essentially invalidate all the results obtained about the relative effectiveness of 
performance contracting. The best way to determine the effectiveness of an educa- 
tional program is to have truly equivalent groins and give just one of them the treat- 
ment while ensuring that the other is relatively unbiased by the existence of the 
experiment. 

A second major mistake is using grade equivalent gains as the criterion of 
jgj^^iccess. Now we all know that such equivalents are misleading fictions based on 



-103- 



questionable extrapolations. As anyone who has taught school has discovered, stu- 
dents do not learn in nice equal intervals throughout the year. There is a big 
spurt in the fall, often to make up for losses during the summer, and then a leveling 
off in the spring. Thus, if a program runs for only a portion of the year it is 
not wise to siinply. extrapolate the results and pay the contractor on the basis of 
what performance might have been over a full academic year. 

A third mistake also is related to payment in that such payments often are based 
on the number of students achieving various grade equivalent gains. Robert Stake has 
presented a rather devastating demonstration of the errors that can result from such 
an approach. This illustration is as follows: 

"Just how unreliable is the performance-test gain score" For a 
typical standardized achievement test with two parallel forms, A 
and B, we might find the following characteristics reported in the 
testes technical manual: 

Reliability of Test A = +.84. 
Reliability of Test B = +.84. 
Correlation of Test A with Test B = +.81 . 

Almost all standardized tests have reliability coefficients at this level. 
Using the standard formula, one finds a disappointing level of reliability 
for the measurement of ijrnprovement : 

Relial)ility of gain scores (A-B or B-A)= +.16.. 

The test manual indicates the raw score and grade -equivalent standard devia- 
tions. For one widely used test, they are 9.5 items and 2.7 years, respec- 
tively. Using these values we can calculate the errors to be expected. On the 
average, a student *s raw score would be in error by 2.5 items, grade equivalent 
would be in error by 0.72 years, and grade- equivalent gain score would be in 
error by 1.01 years. The error is indeed large. 

Consider what this means for the not unusual contract whereby the student 
is graduated from the program, and the contractor is paid for his instruction, 
on any occasion that his performance score rises above a set value. Suppose - 
with tlie figures above - the student exists when his iinprovement is one grade 
equivalent or more. Suppose also, to make this situation siinpler, that there is 
no intervening training and that the student is not influenced by previous 
testing. Here are three ways of looking at the same situation: 

Suppose tliat a contract student takes a different 
parallel form of the criterion test on three successive 
days immediately following the pretest. The chances 
are better than 50-50 that on one of these tests the 
student will gain a year or more in performance and 
appear to be ready to graduate from the program. 
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Suppose that three students are tested with a 
parallel form immediately after the pretest. Tlie 
chances are better than 50-50 that one of the three 
students - entirely due to errors of measurement - 
ivill gain a year or more aixJ appear ready to graduate. 

Suppose that 100 students ara admitted to contract instruc- 
tion and pretested. After a period of time involving no 
training, they are tested again, and the students gaining 
a year are graduated. After another period of time, another 
test, and another graduation. After the fourth terminal testing, 
even though no instruction has occurred, the chanccjs 
are better than 50-50 that two -thirds of the students 
will be graduated/* 

It becomes apparent that a school could pay for many gains that are just due to 
chance. The contractors, however, also have problems in that many students may 
test off the bottom of the scale on the pre- test. This means that they are assigned 
higher pre- test scores than they deserve and at least some of the gain in perfonnance 
may not be recorded. A similar argument holds for students who do very well on the 
pre- test. Perhaps these factors balance out, but if I were a school district or con- 
tractor, I would resolve these problems in advance. In a minutes, I will discuss one 
way of doing this, but first let us consider a fourth error. 

A fourth mistake in evaluation designs is the use of average gain for evaluating 
the total effectiveness of the program. Such an approach is obviously better than 
counting the number of students who reach a given criterion, but it still falls short 
of being adequate. The reason for this is that two groups could achieve essentially 
the same average score by quite different routes. Suppose, for example, that a 
contractor knew he would be paid for only those students who showed a fe<.>: or morels 
growth in grade equivalent. Would it not be to his advantage to focus his efforts on 
just the students who appeared to have the most potential for achieving this goal 
and essentially ignore the rest? The end result of such a capitalistic endeavor 
would be more students for whom the contractor was paid, but the average score for 
the total group would be the game as or lower liian the one for students in a control 
group where attention was distributed equally. This kind of bias would show up as 
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a much larger standard deviation in test performance for the experimental group than 
for the control group. 

I could go on ticking off several more mistakes, but I think I have made my 
point that there are often many evaluation design problems with performance contracts. 
Let me conclude, therefore, with some suggestions of how to design an adequate evalua- 
tion of a performance contract. 

1. Specify the kinds of information you want to gather about the contracting. 

2. List the objectives you want to achieve and their relative priorities, and 
then use this information in selecting and/or constructing appropriate measurement 
tools. Special attention should be given to the availability of objectives banks and 
itemi pools that would facilitate this process as well as improve communication be- 
tween the district and the contractor. 

3. Where possible, select a control group that is truly equivalent and unbiased. 

If this is impractical, it might be worthwhile to see if the experimental group^s 
agression slope between pre- and post-test is steeper than that for a comparison 
group. If it were, then one could cautiously infer that contracting was at least 
having a positive effect although it worJd not be possible to say how much. 

4. Monitor what is happening in both the control and experimental classes to 
ensure that students are not being coached on the answers to specific questions as 
well as to determine just what procedures are really being used so that the critical 
ones can be identified. 

5. Including monitoring of classroom activity Ln the evaluation desi^ is also 
of value for measuring any important side effects, such as relevant teacher and 
student attitudes. 

6. Analysis of results and payment to the contractor should be based on the 
average scores of subgroups of students. These subgroups might be formed by dividing 
the saiTiple into thirds so as to check on whether the contractor is devoting all his 
efforts to just one kind of student. 
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I hope fnese suggestions help you in evaluating the performance contracts you 
have now and the ones you are planning. The reason' I say this is that I know that 
the OEO conclusion will not deter responsible schools from exploring the utility of 
various educational programs that might be offered, whether they come front within 
or outside of the school community. I only hope we can find some good ones to solve 
our educational problems. 



\^NTURES IN PERFORXL^NCE OT^CTING 
By: 

CHARLES BLASCHKE, President 
Education Turnkey Systems, Inc. 

Performance contracting has been hailed as the "hottest thing in education" 
by the news media, "hucksterism" and a "conspiracy by private firms to take 
over public schools" by critics, and a "panacea" and "miracle worker" by zealots. 
Neither the critics nor overzealous advocates do justice to this "managerial 
innovation," limited technically and by the intentions of man himself. The 
objectives of my comments today are several-fold: a) to briefly describe the 
concept of performance contracting and turnkey operations; b) to evaluate it 
in terms of the several criteria for which it was designed to effect change 
and to evaluate some of the recent "headline" grabbing reports, specifically 
the O.E^O. Report; c) to discuss inherent and man-made problems; and d) to 
attempt to project its evolution into the future. 

The Approach 

The performance contract- turnkey approach is a managerial tool designed to 
ensure that results are achieved in a way that encourages responsible innova- 
tion. A school district would enter into a contract with an outside firm or an 
internal teachers* group to accelerate achievement (usually math and reading) 
of a limited number of students (usually ESEA Title I eligible) with reimburse- 
ment to the contractor based on the actual performance of the students measured 
by achievement or performance based tests. After a period of successful demon- 
stration, the school would then adopt or expand the contractor's instructional 
program on a turnkey basis making the necessary changes in order to realize the 
potential cost-benefits of the contractor's program. 

A school district could decide to initiate a performance contracfe-tui:^3ikey 
project for one or all of the following reasons: 
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o 



to provide supplemental capability in a program area where it 



presently does not exist or would be too costly to develop 



internally (e.g., vocational training); 



to use it as a vehicle for testing, analyzing and validating 



newly developed instructional systems in order to determine 
whether or not to adopt or expand them on a wide scale basis; 

^ to assist in sol ving pol itical , social and economic problems 
confronting school administration* 

The heart of the performance contract-turnkey approach is the "performance 
specification"', usually included in a Request for Proposal (RFP) sent to pro- 
spective bidders or local teachers* associations. This document includes not 
only the performance specifications desired, usually in terms of grade level 
equivalents or criterion reference based objectives, but also particular con- 
straints such as the average amount of dollars to be provided per student and 
the student's time available to the contractor. Based upon the RFP, the con- 
tractor's proposed response, and face-to-face negotiations, a final contract 
evolves. If the heart of the concept is the RFP, the life-blood must be the 
turnkey phase. After the project has been initiated for a period of seven to nine 
months, a turnkey analysis is conducted, usually by a Management Support Group, 
the purpose of which is several-fold: 



o 



to determine the relative cost-effectiveness usually in cost per 



some unit of achievement of the contractor's program in mathenatics 



and reading as coinpared to the existing school' 



s program m 



siipilar areas with similar students; 



to determine the economics of the contractor's instructional pro- 



gram for planning the nature and extent of the turnkey phase the 



second year: 
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° to determine the nature and extent as well as the cost of management 
changes that have to be initiated by the school to achieve the pro- 
ject cost and benefits which the contractor has demonstrated could 
be achieved. 

For example, the contractor could guarantee that the school could achieve 
90% of the effectiveness which he demonstrated could be achieved utilizing dif- 
ferentiated staffing, incentive pay and program budgeting techniques if the 
school would incorporate such changes into the turnkey classes. A lesser 
guarantee would be offered if the school decided to adopt the learning system 
with only three days of teacher training. 

Hence, the school superintendent who adheres to this "turnkey" notion will 
be able to consider alternative levels of costs and benefits in deciding the 
scope of the turnkey phase. He can present them to the school board with a 
leverage that previously did not exist. Moreover, the contractor not only dem- 
onstrates an effective program, but also accepts the responsibility of providing a sys- 
tem tliatcan be incorporated into or expanded within the school system on a 
turnkey basis with levels of guarantee. Therefore, the performance contract- 
turnkey approach should not be viewed as an end in itself. Rather, it provides 
a means by which the local school system can experiment in an effective manner, 
have a new instructional program demonstrated and tested in a local environment, 
and adopt the new program on a turnkey basis making changes within the system 
to ensure that the potential results can be realized. Whereas the firm "bit 
the bullet" the first year, the school management must, the second. 

Type of Contractors 

The majority of the projects have been conducted by private corporations, 
some of whom have utilized teaching staff which remain under employment of the 




school district. Most of the firms have had past experience in program instruc- 



-no- 



tion, the use of teaching machineb and contingency management. The vast majority 
of the firms are small to medium size--not by any means the equivalent of the 
military-industrial, social complex! Performance contracting has not attracted 
the large educational firm because their materials in most instances are not 
competitive and the firms are anxious about reducing mark-ups to become compet- 
itive because of the establishment of precedence. Rather than manufacturing 
equipment or software, the firms are generally systems management groups which 
utilize the material equipment which is commercially available and they feel 
will work best with the particular students. The instructional systems 
utilized range from those utilizing sophisticated teaching machines, computers 
for instructional management, prescription and diagnosis, and with high stu- 
dent ratios (e.g., one contractor utilized one professional and 32 parapro- 
fessionals for 600 students instructed in mathematics and reading) to more traditional 
ones. While some firms utilize material rewards, others rely more heavily on 
intrinsic motivation to increase the performance of the students. With the 
seemingly large variances in instructional systems design, several common threads 
appear* 

^ Use of individually prescribed self-paced instructional programs. 
^ Use of proven classroom managemeftt techniques to ensure the best 

use of the teachers' or classroom managers* time. 
^ Use. of para-professionals and differentiated staffing, 
^ Use of programmed texts or programmed software combined 

with audiovisual media of presentation in many instances. 
^ The use of contingency management incentives for teachers 

and students, either extrinsic or intrinsic. 

In 80 to 90 per cent of the projects the firms. guaranteed a minimum grade 
level increase per child or no payment would be made (in the GEO Experiment, 
the minimum grade-level gain was initially set at 1,0). In most instances. 
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incentives were provided for incremental gains above the minimum level; in 
others, penalties were imposed on a prorated basis below a specified level of 
student performance. In those projects in which the participating teachers 
remained on the payroll of the school, but were assigned to the contractor for 
the duration of the project, the contracted fee for raising a student one 
grade level per subject ranged between $45 and $85, with one exception of about 
$300. Payment to contractors providing total learning systems, including loc- 
ally trained personnel to operate the centers, ranged from $81 to approximately 
$220 per grade-level gain in math or reading. 

During the last school year at least two teacher's associations contracted 
with the local board of education on an incentive contract basis whereby 
teachers could collectively earn several thousand dollars, which were to be 
pooled by the Association teachers. In one of the projects, staff different- 
iation was utilized in the classroom; in the other a highly individualized 
program of instruction was utilized. 

Several projects implemented this year will provide incentives for 
Individual teachers and even parents based upon student performance. In a 
USOE sponsored project in four sites, teachers could earn up to $1,200 per 
class and parents $100 per child, based upon performance above the class ex- 
pected gain. In Wethersfield, Connecticut parents can earn $20-$30, if their 
child masters prescribed behavioral objectives.- In Dade County, Florida 
teachers can earn as much as $110 per student for gains above expected levels; 
moreover, teachers will also be provided $55 per student to defray operating 
costs and have the option to utilize $55 per student as risk capital to invest 
in the classroom with the contingency that if a student's performance is not 
above the expected gain, all $55 has to be returned. Briefly, performance 
contracting is a problem-oriented concept flexible enough to be applied to a 
ERJCjmber of areas by a number of potential "contractors." 
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How Did It Fare? 

Any final evaluation of performance contracting and turnkey operations at 
this time would be prema+'jre--at least this was a general consensus of a 
recent meeting of AERA-AASA Federal officials and experts five months ago. Im- 
pressions can be gleaned from on-site observations, the Rand study (which 
is being extended) of five non-OEO sites, and the much publicized OEO Prelim- 
inary Report on its experimental effort. With your permission I should like 
to suggest the following criteria included in the original Texarkana project, 
explicit in my letter to Secretary Finch in January, 1969, a) to provide a 
more cost-effective approach in areas such as math and reading; b) a low- 
risk, low-cost vehicle for experimentation; c) a means for increasing inno- 
vation; d) a catalyst for school system renewal through the turnkey concept; 
e) an opportunity for increasing community participation; f) a politically 
acceptable and educationally effective means to integrate or provide equity 
of results; and g) an opportunity to rationalize the collective bargaining 
process. 

Cost Effectiveness 

One side of this criterion is results achieved by students— measured by 
standardized tests, criterion-referenced tests, performance objectives , or 
other instruments. Prior to the preliminary release by OEO, most projects 
indicated that student achievement rates were just about doubled compared to 
previous years progress which was usually 0.3 - 0.5 grade gain. The Rand 
Study of five non-OEO projects indicated that student gains ranged from 0.4 
to 1.7 grade gains. A recent reassessment of the results of the seven pro- 
jects in the Virginia demonstration indicates that they might have been 
understated by as much as one grade level, as a result of test-curricula mis- 
matches noted in the Rand Study. Also, *nower-than- 75-IQ students" did better 
than those with higher IQ's. And then we have the OEO results, which were 
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disappointing to all of us involved in implementing the project. However, 
equally disappointing was the manner by which OEO reported these results. 

Across-all -site comparisons of grade level differences between experimental 
and control groups, which showed few significant differences, are not that 
"interesting" when one considers the original OEO objectives: "which of 
the 18 instructional programs, if any, would work best." After probing by ' 
the press, OEO officials did note that four-five statistically significant successes 
for each failure did appear in small to medium-sized South and Southwestern 
schools, as reported in the Battelle Memorial Institute interim report to OEO. 
These schools were less rigid and unionized compared to Northeast and Western 
schools where control students did relatively well at the junior high level 
(e.g., .8 to 1.0) compared to previous rates of growth, possibly due to 
teacher inspiration induced by the private contractor. 

Probably the most significant variable in predicting success was the school- 
firm interface problem (See Illustration I). Where they arose during the first 
months or so after school began, the projects had very little hope for success. 
Where contractors' teachers ("scabs") broke the "picket lines" during strikes, 
organized teachers never forgot performance contracting; "fisticuff^' between 
firms' and schools* representatives are not indications of a close working 
relationship; bad pre-test conditions, created by last minute scheduling, and. 
several days of testing didn't engender good feelings of principals toward 
the project; and programs just don't operate effectively when high level cor- 
porate and school officials are threatening law suits and contract terminations. 
An "eyeball" analysis of Illustration I speaks for itself. 

The preliminary results released by OEO will probably be criticized by 
GAG, the agency's watchdog, and others regarding: a) the short lead time and 
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SCHOOL-FIRM INTERFACE * * 
IMPACT UPON RESULTS 
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ILLUSTRATION I (Continued) . 



The preceding chart displays the interface variables at project sites which were 
not necessarily taken into consideration by O.E.O. or Battel le in their evaluation 
of the O.E.O. Performance Contracting Project. Substantial evidence exists at the 
O.E.O. Project Office, School Site Project Offices, Sub-Contractor Offices, and in 
the Project Documentation System to verify the existence of these conditions. Anyone 
undertaking a review of these conditions will be required to not only determine their 
severity but to also assess the degree to which each condition may have contributed 
to the evaluation findings as reported by Battelle and O.E.O. The accuracy of any 
evaluation is only as good as the quality of the data considered and the particular bias 
of the analysts, especially in light of the political ramifications of this project. 

All of the above problems which occurred for the most part during the first four 
months of the project had a lasting impact on the project^ the attitude of these affected 
towards it, and the final results of the experimental and control schools student 
performance. 

DEFINITIONS: 

Column 1: TEACHER RESISTANCE - includes non-acceptance by teacher organizations 
at sites (e.g., strikes, grievances filed, lawsuits, etc.) within project schools, 
and by key individuals (principals, teachers, administrators) operating within the 
framework of .each control and experimental school . 

Column 2: MANAGEfCNT CONFLICTS - includes serious personality conflicts between 
representation of the firm and school officials such as the project director or 
principal; turnovers of management personnel because of interpersonal relations 
or incompetence. 

Column 3: POOR TESTING CONDITIONS - reflects the poor pre-test conditions (usually 
resulting from short lead time and inadequate planning and scheduling) as stated 
in the Battelle Evaluation Report by the Battelle Memorial Institute people who 
administered the tests. 

Column 4: THREATS OF CONTRACT TERMINATION OR MAJOR RENEGOTIATIONS - includes 
formal threats of contract terminations from O.E.O. to School Districts and educa- 
tional finns, from School Districts to educational firms, and from educational 
•''irms to School Districts as a result of firm-school problems and non-compliance. 

Column "E": The number of experimental grades in which experimental students did 
better than control grades based upon the Battelle analysis. 

Column "C": The number of control grades in which control students did better than 
experimental grades based upon the Battelle analysis. 
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lack of school involvement in planning, a criticism rendered earlier by the KEA 
and recently by the 18 project directors in their report to OEO; b) test con- 
ditions and instrumentation, recognized as a caveat by OEO in their report (e,g. 
bottoming-out effects or low correlation between pre- and post-test results; and 
others. These criticisms will certainly add to the controversy and cast some 
doubts on whether performance contracting as a technique of instruction for 
producing achievement results did get a fair test. 

Cost of OEO Experiment 

The costs of the 12 performance contract projects analyzed by Education 
Turnkey Staff using for the first time the COST-ED Model, are rather revealing. 

First, while many firms used similar materials, the economics of the sys- 
tems varied significantly, especially regarding staff use, equipment, books 
and audio-visual costs. For example, in the control sites about 70-75%, and 
1-2% of total costs were spent on teacher pay and books and audio -visual 
method materials respectively; the contractors spent 55-15^ in the corresponding 
areas . 

Second, compared with control programs, contractors^ investment in in- 
structional equipment i*? significantly greater in most programs. 

Third, if schools adopted contractors* instructional programs, operating 
costs would be less than existing school costs per student/subject in over one- 
fourth of the cases and somewhat greater in the rest. 

Fourth, achievement scores in contractors' programs will not have to be 
significantly greater than control program scores for contractors programs 
to be more cost-effective than the schools, assuming that the average control 
scores were .5 grade equivalent gain, the contractor would have to produce the 
following results to be equally cost-effective: 
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Alpha Learning : 



0.53 



Quality Ed. Development : 



.63 



Learniiig Foundations : 0.82 



Singer-Graflex : 



0.57 



Plan Ed. Center: 



0.67 



Westinghouse Learning : 



0.53 



The reasons for variances and lower than expected costs noted in the Turnkey 
Report to GEO included: 



Lower classroom costs through better student scheduling and util- 
ization of facilities, space and instructional equipment. 

Lower staff costs through the use of paraprofessionals to operate 
self-paced> individualized student learning systems. 

Reliance on instructional components with relatively low operating 
costs, such as teaching machines, cassettes, and non-consumable 
programmed instructional packages. 

Better management control and greater administrative and classroom 
flexibility t! an in traditional settings. 



Aside from the relatively high start-up costs involved In performance contract 
projects, a primary consideration must be the public ^s present attitude towards 
school costs and where costs should be cut. The Gallup Survey of Public Atti- 
tudes Toward Public Schools (See September, 1971, Phi Delta Kajjpan ) disclosed 
that the number one problem facing schools is "finances""-where should costs be 
cut when local boards are forced to reduce total budgets? John Q. Citizen does 
not want to increase class si2e (79% oppose) or to cut teachers* salaries 
(77% oppose) but would want to reduce the number of administrators (50% favor) 
or the number of counselors (32% favor), for example. The general public is 
either emphatically certain about what constitutes good education policy and 
contributes most to student achievement, or is totally ignorant about the 
economics of school operations and budgets. An analysis by Education Turnkey 
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Staff of the typical secondary school's costs (elementary math) derived from 
national averages used in the OEO study is illuminating: 

That the savings incurred through renting books rather than 
providing them free of charge could be surpassed by increasing 
class size by one student or by reducing the average annual 
pay of teachers (e.g., by hiring paraprofessionals or younger 
teachers) by an amount less than 1% of the total budget. 

That a decrease in annual pay of teachers by S% will free enough 
resources to increase audio -visual materials and books by 

COST-ED Analyses of these and many other equal-cost trade-offs 

indicate the cost saving potential of performance contracting 
In the same survey, 49% of the public favored performance contracting; however, 
the public's attitude towards the cost saving implications could constrain 
the effective adoption of performance contract learning systems during the turn 
key phase. Public perception will change as educational myths and concepts are 
displaced* , 

Low Risk-Low Costs Means for Experimentation 

A second major index for evaluating performance contracting is whether it 
did provide a low risk, low cost way for districts to experiment. Because 
many of the firms were overly ambitious or optimistic in terms of grade-level 
guarantees, the actual fee paid by the school system in many cases was small 
relative to the increases in student performance. One district, for example, 
paid a fee less than existing school costs for a doubling of the rate of 
learning. Schools also avoided risk: in most instances, the political "heat^' 
resulting from the experimentation was not directed toward the school but to 
Federal sponsoring agents or to the performance contracting firms — (e.g., the 
BRL project in Gary, Indiana). Similarly, in those instances where the con- 
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tractors' results were not significant, {See Charles Blaschke, Performance 
Contracting: Mho Profits Most ? ( Phi Delta Kappau , Bloomington, Indiana, 1972 ; 
also see Charles Blaschke, " Performance Contracting, Costs, etc .", Phi Delta 
Kappa n , December 1971.) the contractor again, rather than the district "failed/' 

The Virginia Department of Education in its report to the State Board on 
lis performance contract project in seven districts expressed dismay at the 
gains made on standardized tests (since proven to be an understatement), but 
noted; "The use of performance contracting as a method for delivery of an 
instructional program cannot be deemed a failure on the basis of results in 
Virginia .... As experienced here performance contracting, as a means for 
low risk, low cost experimentation in education innovation can be considered 
successful." 

Increased Innovation? 

Performance contracting was also designed to encourage responsible inno- 
vation by prescribing levels of performance and costs constraints, but not 
the methodology or materials to be used by the contractor. During the first 
year, the most significant Innovation was the design and actual application of 
"total learning systems." In this respect, performance contracting did allow 
the flexibility for firms to " systems engineer" a variety of methodologies 
and curriculums into learning systems which were tailored for the target popu- 
lations. 

With the exception of the first Texarkana project, the new EMR project in 
Grand Rapids, and a limited number of others, few radically, innovative learning 
systems, hardware or software developments, or pedagogical approaches have 
surfaced. Perhaps, the lack of development funds for performance contracting 
has been a significant factor. Or perhaps, there is dawning a realization that 
Q classroom instructional management rather than '^gadgetry" might be more signifi- 
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cant in producing results. 

A Catalyst for Reform 

A primary criterion ^^^st be the impact of performance contracting on school 
system renewal. Before achievement scores were available, about a third of 
the schools involved in performance contracting in 1970-71 planned to continue the pro 
jects next year; another third plan to adopt on a turnkey basis the contractor's 
program in part or totally; and the rest are undecided. One Virginia site 
expanded the turnkey phase from two schools last year to 10 this year; all three 
projects in Grand Rapids are being continued or turnkeyed and an additional 
project in special education will be initiated. A turnkey operation at the 
elementary level has been implemented in Taft, Texas. In 70 to 80 per cent of 
the turnkey projects, local rather than "non- formal" federal funds are being 
used. That turnkey projects v/ill be operated as effectively or efficiently 
as last year's performance contract projects is uncertain. Only the results 
a year from now will tell —if school administrators are willing to initiate 
management changep and Independent evaluations are performed. 

Alleviating Political and Social Problems? Was Performance Contracting De-Humanizing? 

One of the serendipities observed over the last two years has been a unique 
psychological reversal in the classroom--namely, the firm, the teachers, and 
others are dependent, monetarily or otherwise, upon the success of the individual 
students. In several projects teachers began to perceive themselves as "learning 
and resource partners." Instruction in this sense was not only "learner centered," 
but also "learner controlled." The impact of the latter teacher and student 
attitudes might! have been significant. (See "The Grand Rapids Story", film 
ed. J. Mecklenburger, Indiana University.) 

Although the teachers* attitudes toward the projects ranged from extremely 
negative to extremely positive, the majority of the teachers felt that perfor- 
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mance contracting did allow them, (within certain limits) a degree of flexibility 
to do what they had always wanted to do. In certain sites, participating teachers 
have become "salesmen" for performance contracting within the school and in the 
inmediate area. Early involvement of teachers during planning was critical to 
positive teacher attitudes and cooperation and at the least non-disruptive 
project operations. 

Student reaction to the project has been observed in several areas. A 
"smile factor" was noticeably prevalent in many projects; attendance was 
generally significantly higher than in control sites ( through the availability 
of make-up classes, actual attendance in one performance contracting site was 
greater than the number of regularly scheduled hours available); and dropout 
rates were significantly reduced in the vast majority of sites analyzed thus 
far. In one Virginia project involving 500 students, the dropout rate of the 
target group fell to zero. 

Did Community Involvement Increase? 

A New York City District viewed performance contracting as a leverage not 
only to countervail union pressures but also to involve community residents as 
paraprofessionals and teacher aides^ In Taft, minority parents threatened 
to withdraw their children from the project, arguing that inferior aides were 
teaching their children and that segregated classes were being perpetuated. 
Over time, as communications between the school and the cormiuntty increased, 
parents* resistance subsided. 

At Dallas where disciplinary problems were about to force discontinuance of 
contractor's program, parents who had been members of the planning advisory 
group formed voluntary parent committees which patrolled the school hallways to 
ensure that the project could be continued. 
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In the majority of the projects, principals reported that a high level of 
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parental support prevailed during the entire year, even though a few parents 
withdrew their children from the program during initial stages. 

Did It Rationalize the Collective Bargaining Process? 

Witliout a doubt performance contracting has provided a leverage for school 
administrators trying to initiate incentive or merit pay, differentiated 
staffing, and even "profit-sharing" arrangements. In one performance contract 
site, the school board plans to initiate incentive programs for -^11 students 
and teachers during the turnkey phase. In other sites, school principals 
have attempted to initiate incentive contracts with their teachers in a manner 
similar to that in the performance contract school. In at least one of the 
two projects, suits were filed by the teacher's groups resulting in the discon- 
tinuance of incentive pay during the last semester. 

Was It an Aid to Desegregation? 

While it is too early to judge, performance contracting does seem to be 
considered an aid to desegregation. For example, the NAACP recently passed a 
resolution favoring performance contracting. One performance contract in a 
Southern state last year was funded under the Emergency School Fund Act. And 
the presence of performance contracting in Texarkana over the last two years 
has not only soundly defeated freedom-of-choice advocates at school board 
election time (Texarkana is the hometown of Freedom, Inc., the national advo- 
cate of "freedom of choice"), but also has enabled integration to occur 
relatively smoothly in Texarkana, Ark., while race riots occurred in the non- 
participating district across the street in Texas. Neighboring Dallas is 
justifying its "desegregation by TV" on the concept of "equity of results", 
a performance contracting spin-off. 

In several sites where administrators looked upon performance contracting 
as a means to assist desegregation, recent court orders and decisions required 
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the closing of schools or transferring of students which affected the validity 
of any evaluation. 

New Directions: Problems and Potential 

Originally conceived as a catalyst for school system reform, first- 
generation performance contracting by private firms should put itself out of 
business for the most part within the next couple of years, not because of 
its failure but because of its success. While school officials will continue 
to use it as a low risk, low cost vehicle for experimenting with radically 
new or untested learning systems, its major contributions will have been made 
in the iminedlate future. 

•k 

Performance Support Contracts 

As the results of learning systems used by contractors become available, 
both contracting firms and schoolofficials will see the advantages of entering 
into turnkey projects irmiediately, without going through the costly and time- 
consuming performance contract stages. Previously sold only materials, schools 
are now getting training and other support from firms with guarantees (e.g. 
external contracts in Dade County). With cost effectiveness data available 
for a large number of learning systems, it is possible to simulate the cost' 
effectiveness of alternative programs under varying constraints to assist 
officials in selecting programs or reducing their costs. We are conducting 
such simulations in Michigan at the present time for the Department of Education 

* Excerpts from Charles Blaschke, Performance Contracting: Who Profits Most? 
( Phi Delta Kappan , op. cit.) 
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Performance support contracts are presently under way in Chicago, Detroit, 
and Miami. In the first two districts, LRA and Alpha Learning II, respectively 
are providing teacher training, materials, equipment, and monitoring services 
to both principals and teachers; the major risk is assumed by the firms even 
though the teachers remain under the employment of the districts. In Miami, 
Behavioral Research Laboratories, operating under similar conditions, will 
receive their maximum payment if elementary students achieve TOO per cent 
above the expected gains and none for gains less than 10 per cent. The major 
problems anticipated in such contracts include illegal delegation of authority 
to the firm regarding supervisory and firing or transfer policies, and conflict 
with union and school regulations regarding teacher working conditions and 
maintenance liability (for example, if the district purchases the firm^s 
equipment, then the school's maintenance personnel are required to service the 
equipment, and any downtime affects the firm's costs). Even these potential 
problems are less formidable than those in first-generation performance contracts; 
guarantees by firms are likewise less extensive because of lack of management 
control . 

Performance Pacts Between State Departments and Local Districts 

The idea of contingency funding and "grants management" between funding 
agencies and districts has been batted around at the federal level since 1966 
when Bureau of the Budget officials proposed to the S. Office of Education 
that Elementary and Secondary Education Act Title I funds be based upon results 
achieved. Departments of Education in several states have discussed and con- 
sidered accountability "agreements" with locals. None, however, were implemented 
until November, 1971, when Michigan initiated its $23 million accountability 
model, possibly the most significant turning point in public education during 
the century. 

Sixty-nine districts have been awarded amounts ranging from $6,000 to over 
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$11 million to increase achievement of minority students scoring below the sixteenth 
percentile in math and reading. The districts have been given specific achieve- 
ment levels as goals. If, after the first year, tests indicate that each 
student achieves the specified level, the district receives the full amount of 
funding the succeeding year based on the state formula. If, on the other hand, 
the students achieve less than 75 per cent of the specified goals, a prorated 
penalty is applied. 

"Revolutionary" hardly describes the project. First, the districts receive 
in essence a fixed fee per student to raise him to a specified level or be 
penalized the following year— grants management at its highest level! Second, 
districts that are most efficient in meeting the objectives will be rewarded, 
since the amount of the fee is based upon results, not costs incurred. In 
this respect, the project differs from the vast majority of federally funded 
projects. For example, given a fixed fee of $200 per student, a district 
could purchase a system costing only $50 per student; if it produced the necessary 
achievement level of 0.8 years growth, a $150 "profit" could be earned and 
used for general and administrative purposes by the district or $50 could be 
shared with the faculty as a whole!! Third, each district now nas an incen- 
tive to search the market place for the learning system which it feels will 
produce the necessary results at lowest cost. Cost considerations have often 
been neglected in per-^ormance contract projects funded with federal 
funds. One firm*s fee for raising a student one grade level was 80 per cent 
above the school's existing cost to produce similar resulcs. And last. 
It could put the districts out of the compensatory education business as the 
number of eligible deficient students decline, if the district does its job 
right; it could be put out of business altogether if it does not, as state 
aid dries up. 



Implications for performance contracting between districts and teachers or 
private firms are significant. For example, certain teacher groups are pres- 
sing for a "profit-sharing" arrangement with the administrative offices if 
students acliieve prescribed levels. Of the $23 million, $500,000 is specifically 
earmarked for contracts with private firms; a large number of districts are 
entering similar performance support contracts with private groups with the 
$22.5 minion. In Detroit, it is estimated that several million dollars 
will be allocated to perfonnar.ee support contracts. 

As with any bold and innovative undertaking, the Michigan project inherits 
some of the problems inherent in performance contracting. First, even though 
officials are hopeful that criterion referenced tests will be used, most 
districts will propose to use norm referenced tests, which will require state 
approval. Since individual rather than mean scores will be the basis of 
determining future allocations, the standard error of most norm tests will 
take its toll on the districts. Second, because teachers will administer 
tests and will be aware of the specific tests to be used, the opportunity 
for teaching to tests exists. Allegations, just or unjust, are certain to 
be made. And third, violations of USOE "comparability guidelines" could be 
at conflict with systems used by the school districts. 

Incentive Contracts With Teachers 

In two of the Office of Economic Opportunity project sites. Mesa and 
Stockton, the districts entered into contracts with their teacher associations, 
whereby the teachers received incentive payHientS:»tes«d;>^ student performance. 
The participating teachers chose to pool the incentives earned during the 
year. In the majority of states, such contracts would be illegal, since 
teacher association charters do not specify such activity and services; waivers 
were required in the OEO project. 



U -127- 



Because of legal and political problems, most teacher incentive projects 
have taken on a new character. The most innovative has been developed in 
Miami. Beginning in March, 1971, representatives of three teacher associations, 
parent groups, administrators, and students formed a Professional Advisory 
Committee, (See Charles Blaschke, "Performance Contracting Newsletter", 
Nation's Schools , May, 1972.) PAC, to assist and advise in the development 
of a Request for Proposal. Specifications discussed at a prebid conference 
included the following: 

The faculties and firm could receive up to $110 if students 
average a grade gain approximately 100 per cent above expected gains in math 
and reading as measured by standardized tests and "banks" of performance 
objectives. 

Contractors received $55 per student to be used to defray operating costs, 
without the risk of having to pay back the amount. Faculties could invest 
up to $55 per student of "risk capital" for teacher training or instructional 
classroom equipment; however, if the students achieved less than 50 per cent 
above expected gain, a portion, if not all, of the risk capital expenditures 
would have to be repaid. 

Technical support was provided to the interested faculties by Turnkey 
staff, administrators, and representatives of the teachers associations. Pro- 
posals submitted by teacher groups and five private firms indicated that 
teachers were willing to guarantee a higher level of student performance than 
the firms. However, the teachers demanded certain quid pro ,quos from the 
district, such as 24 hour maintenance service, specific information regarding 
student achievement levels and validation results of instruments to be used, 
and greater classroom flexibility. Moreover, the teachers are using the risk 
O , capital allocations and negotiated agreements with equipment suppliers so that 
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the suppliers share the risks, as in a performance support contract. On their 
own, teachers decided to use teaching machines; provide student Incentives; 
and use peer tutors instead of aides; and to Increase class size from 25-1 to 
45-1. 

While legal and political problems of delegation exist, they are minimal, 
especially In light of the participatory management process which was followed 
in the creation of PAC and the development of the RFP* Aside from establishing 
precedents in the use of risk capital and new testing instruments, the project 
is the most visible example of combining incentive contracting with professional 
self-governance, a much discussed goal of the NEA. 

Another variation in incentive contracting with teachers is the USOE-spon- 
sored Project in the Use of Incentives being conducted in San Antonio, Oakland, 
Jacksonville, and Cincinnati. Teachers can earn up to $1,200 if student 
achievement, as measured by standardized tests, is three to four months ahead 
of expected gains. In the first two sites, incentives up to $100 per child 
can be earned by parents. The major objective of this evaluation project is 
to determine whether incentives offered to teachers and parents will result 
in increased student performance for poor, minority group elementary students. 
A second objective is to determine what, if anything, teachers and parents 
will do differently to ensure maximum student achievement. (See Illustration II 
for a brief description of Incentive projects). 

Long Run Impact 

In the long run, to the extent that performance contracting results are 
favorable, it should put itself out of business for the most part as school 
districts Internalize contractor's programs through the turnkey operations. 
However, to the extent that private and public groups continue to develop 
new learning systems which offer promise, performance contracting will be 
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utilized as a low cost, low risk vehicle for experimenting with limited 
demonstration programs to determine the cost-effectiveness of such learning 
systems. At the same time, as new firms enter the field or personnel from 
existing performance contracting form new corporations, schools will demand 
the utilization of performance contracting in new endeavors associated with 
the groups. Similarly, to the extent that performance or incentive contracts 
are negotiated between school boards and teachers' groups, risk capital 
allocations will be increasingly provided to school teachers who are willing 
to risk investing in themselves or the classroom to ensure the greatest 
educational return for the dollar expended. With all the pressures to reduce 
school costs, teachb." groups will propose incentive contracts based on a 
"cost saving" sharing plan. For example, in the typical school 

if class size is increased by 1.2 students and maintenance 
costs are cut by 30^, then teacher salaries could be increased by about $600; 
under the plan, the teacher might demand a bonus of $300 or of the savings. 
To the extent that incentives are made available to districts (e.g., the 
Mondale-Stevenson Bill) teachers will negotiate profit-sharing demands from 
administrators. In short, bargaining will focus more on productivity 
issues. 

Closing Conrment 

Performance contracting and turnkey operations, as conceived by the author 
in 1965, have been applied and are now a reality iji public education* However 
like other educational innovations, as an idea moves from the conceptual stage 
through application and then to expansion, bastardization occurs if not in the 
conjugal bed, at least during the toddler's stage, resulting in applications 
which sometimes are not only unrecognizable but also seemingly contradictory. 
While such is the case in any field, the barriers to innovation in public 
education are significant. 
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The first two years In performance contracting have been both heartening 
and disheartening as well as encouraging and discouraging for tliose involved. 
Moving from promise to performance has not been as easy a task. While perfor- 
mance contracting has encountered many of the same difficulties confronting 
any educational innovation, "never in the course of public education have 
so few with so little done so much to threaten, unjustifiably, I feel, so 
many." 



ERIC 



PERFORNiANCE CONTRACTING IN DALLAS 
By: 

ROSCOE C. SMITH 
Former Ass*t. Director o£ Accountability 
Dallas Public Schools 

First o£ all, my I ask what would you do if you had five (S) high schools located 
in a low socio-economic area that consists of 1.2% American Indian, less than 3.4% 
White, 85% Black, less than -01% Oriental and 10.4% Mexican American, and these stu- 
dents scored 2 or more grades below level together with a projected drop-out rate of 
30% 

Well^ I will tell you what we did. We gave these students free school supplies, 
spent thousands of dollars on Math Labs and Reading Kits, carried these students to the 
symphony. Opera and the civic ballet. We also had the Dallas Theater Center produce 
plays, scheduled professional actors into the classrooms to prepare these students for 
the performance and then bused them to the theater center to see the production. 

In short, we spent thousands' and thousands of dollars to no avail. Students still 
dropped out of school. Students were still reading on the fourth and fifth grade levels. 

At this time, the decision was to get the most for the money. Therefore all of our 
Title I funds were concentrated in the primary grades. 

We still had this problem. How in the world can we make a dent in this projected 
30% drop-out rate? How can we raise a high school student's reading level above the 
fourtli and fifth grade? 

Dr. Estes, the Dallas Superintendent of schools, began to cry and cry loud. We 
need to provide the best possible education for all of the students. 

A request for proposal was sent out and the New Century Corporation of New York 
agreed to take the responsibility of the Communication and Math Program in this area 
and guarantee the Dallas Independent School District that they would raise these boys 
and girls reading and mathematics more than one grade level this year or you do not pay 
anything. 




This began to ring loud and clear- 



The Thiokol Chemical Corporation said - ^Tick out your potential drop-outs and give them 
to us in a class called Achievement Motivation and we will guarantee that they will 
attend school just as much as the other students". 

We said OKI We identified the potential drop-outs in May and by the first of 
September - over 50% of these students had already dropped out of school. 

We scheduled the remainder of these students into Achievement Motivation and at 
the end of that year they attended school more than the students that were not potential 
drop-outs . 

You know how it goes, people from low income areas have been trained for jobs, but 
still could not find eiq)loyment. The Thiokol Chemical Corporation said, 'TVe will 
guarantee the Dallas Independent School District to train these boys and girls, and we 
will also guarantee to place them on a job or you do not owe us anything." 

WHAT WOULD YOU DO? 

In short, after one year of performance contracting and thousands of dollars in 
federal funds spent, all reports went out that performance contracting was not success- 
ful in Dallas, Texas. 

In light o£ the OEO report, I feel obligated to give a summary of the results in 
connection with Dallas. May I start by saying that performance contracting, en an average 
in the 18 sites which participated in the Office of Economic Opportunity (OEO) per- 
formance contracting experiment, was no more successful than regular public school 
programs in improving student achievement. At some sites, performnce contracting 
appeared to work quite well; at others, however, it appeared to work quite poorly. 

Although performance contracting based on this experiment did not emerge as a 
panacea for curing the ills of education, it did provide in some districts a successful 
alternative approach to the teaching of reading and mathematics at the primary and 
junior high levels. Six companies installed their programs in 18 school districts 
throughout the United States. Quality Education Development was utilized at Rockland, 
ER^C^e; Anchorage, Alaska; and Dallas, Texas. Other companies involved in the program 
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were Westinghouse, Alpha, Singer Graflex, and Plan. OEO did not compare results among 
districts or conipanies. OEO did, however, make available to all project managers a 
printout o£ the evaluation test results. 

These printouts indicated that the results o£ the Dallas program were superior to 
the results o£ programs at the other locations. For example, Dallas was the only site 
to have no negative comparisons (i.e., in all measurements the treatment group in 
Dallas made equal or superior gains to the control group) . 0£ the twelve gain scores 
measured (reading and math for grades 1, 2, 3, 7, 8, and 9), ten treatment -group scores 
were higher than the control groups and the other two were equal to the control groups. 
Furthermore, six o£ the twelve measurements significantly favored the treatment group. 

The math program appeared to achieve superior results to the reading program. At 
all grade levels, the treatment group surpassed the control group in math gain scores. 
In grades 2, 3, and 8 these differences were significant at the .05 level of confidence. 
In fact, at grade 3 the treatment group achieved six months more gain on the average in 
math than did the control group. Reading gain scores for the treatment group were 
significantly higher at grades 1, 3, and 8, again at the .05 level of confidence. 

The final report from OEO concerning the Dallas project states: "Performance con- 
tracting appears to have worked somewhat better with the experimental group than tradi- 
tional classroom methods did with the control group." The report goes on to state that 
the significant differences in gains in the lower grades show conclusive superiority 
for the treatment group in that pre-test differences are too small to account for the 
differences in gains. 

There are other possible reasons for the success of the OEO project in Dallas: 

1. Outstanding management support from Educational Turnkey Systems of Washington, D. 

2. Sandra Malone, one of the best educational analysts in the country, supervised 
the complete operation. 

3. Conplete cooperation from the principals involved. 

4. Teachers involved were outstanding in their performance and dedicated to the 
ERJC cause. 
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5. Don Waldrip, Assistant Superintendent -Accountability, with his positi\e atti- 
tude, motivated everyone he came in contact with. 

The High School Performance Contracting Project was located in five Title I high 
schools. Reading and math programs developed by New Century Corporation were implemented. 
The Thiokol Chemical Corporation placed Achievement Motivation and Vocational Training 
in the "areas of machine metals, automotive mechanics , and drafting for girls, into 
operation during the 1970-71 school year. 

Progress in the direction of the first long-range goal, to increase the academic 
achievement and skill development of students who are educationally deficient, was 
acconplished to a degree. The evaluation would indicate that skill development was 
more successfully acquired than was academic achievement, but gains made by reading and 
mathematics students were generally, although not statistically, better than gains made 
by control students. 

The second long-range goal cannot be adequately measured in one year. However, a 
large portion of the achievement motivation students - students who had been labeled as 
probable drop-outs - remained in school the entire year. Ninety-one percent of those 
students who entered the program in the fall were still enrolled at school's end. 

The third long-range goal, which related to cost-effectiveness, is another matter* 
Perhaps in the long run, because of the infomition gleaned from this experiJDent, cost- 
effectiveness will be increased. High school reading cost the District $374.00 per 
student per year and mathematics, $442.00 per student per year. These figures would 
have decreased significantly if interim performance objective test results had not been 
a part of the payoff formula. New Century received some payment for no gain on achieve- 
ment tests if the student receiving no gain did well on interim performance tests (IPO's). 
If a student showed a gain of . 5 years but still scored more than 75% on his given 
interim performance objectives tests, New Century would still receive $66.61 for that 
student's performance. One hundred and twelve students in mathematics and 96 students 
in reading earned some payment for New Century even though these students gained less 
^han one year in achievement. If one year had been set as the minimum acceptable gain 
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score and interim performance objective tests had not been used, the District could have 
saved $11,249.02. However, for the company to recoup its costs, the students M)uld need 
to have averaged 1.5 years gain on achievement tests and 751 on interim performance 
objective tests • 

In many cases no correlation was found between scores on the two types of tests. 
In fact, in some cases, the correlations were negative. Many possible explanations 
loom; the items were too easy; the achievement tests did not measure the objectives 
of the program; the students are '^turned off by standardized tests. In any event, 
whatever the correct explanation, the audit manager believes that interim performance 
tests play an important role - as a check on th^ progress of the program, to name one - 
but payment should not be attached to them. Criterion reference tests, such as the IPO^s 
could be associated with payment if given at the end of the program, but contractors 
will see to it, out of self interest, that interim checks are made. 

Performance contracting as a means of utilizing the skills and resources of the 
private sector is a viable alternative to traditional methods of developing curriculum 
and staff. The concept of accountability is more readily brought into view inasmuch as 
the contractor earns to the extent that his instructional system proves effective. But 
an in^ortant point arises here. The experiment over the past year has not evaluated 
performance contracting per se; rather it has evaluated certain instructional systems. 
As is the case in most research, some systems work better than others. The effectiveness 
of a particular system which happens to be the product of a company engaged in per- 
formance contracting says absolutely nothing about performance contracting itself as 
a concept. 

For Achievement Motiviation, attendance was measured not in Achievement Motivation 
classes but, rather, in the regular academic classes of the achievement students. 
During the 1969-1970 school year, the target population had averaged approximately' 73% 
attendance. During the 1970-1971 school year, on the other hand, those target students 
who were enrolled in achievement motivation attended school from 84? to 86% of the time. 
Q The Research and Evaluation Branch of the Dallas Independent School District 
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developed a Deportment Scale for regular teachers to use in evaluating the attitudes o£ 
their students, 0£ a possible 70 points, the achievement motivation students received 
average ratings o£ 44.45, while the control students averaged 47 points. But, again, • 
the difference in aptitudes prevented the evaluation from generalizing on these findings. 
Approximately 91% of these students remained in school the entire year - a commendable 
percentage considering the fact that these students were identified as probably, not 
possible, drop-outs. 

Vocational subjects were offered in three areas: automotive mechanics, drafting, 
and m^ichine metals. All progi'ams were individualized, with much of the curriculum 
on tapes. Students would go from entrance to helper level, to assistant level j, to 
apprentice level, to on-the.-job graduate. 

By program 82% of the automotive mechanics students reached a level of employment; 
90.4% of the drafting students reached a level of employment; (helper level or 
above) . 

Many of the employers wrote letters of conffnendation on the students who received 
employment in their establishments. 

The Council of the Great City Schools performed management support services. ^' 
These included developing a management information system and a cost benefit analysis. 
The Council did not feel equipped to perform these services alone; therefore, it sub- 
contracted much of its work out to a conpany named Government Studies and Systems (GSS) . 
GSS developed a management information system under a system entitled RAMIS (Rapid 
Access Management Tnfomntion System). The total cost of the contract with the Council 
was $59,810.00. 

GSS was extremely slow in the delivery of products; consequently the Council was 
late in meeting its contractual obligations to the Dallas Independent School District. 
The Management Information System was not totally operative until the new school year 
had begun. The report is now in the hands of the Dallas Independent School District. 
^ A task of management during the present school year is to modify RAMIS to better 




: the needs of th^ District. The District needs one management information system. 
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not a different one for each operation within the District. 

Inasmuch as all reading and mathematics participants did not make positive gains, 
the computation of costs per learning unit was confounded. This computation, which was 
the responsibility of the management support group, was made by assuming that all stu- 
dents making a negative gain made zero gain. In reading, of those students for which 
both pre-and post- test scores were available, 213 made some kind of gain and 181 did 
not. Using 1.0 as the base performance unit, it would require $374.00 invested per 
students for each one year growth in the program. 

In mathematics, 188 of 319 students made some gain, while 131 made no gain at all. 
The cost of this program was $442.38 per student/year growth. Of course both of these 
figures would decrease if more students could be moved from the negative gain column to 
the positive side. 

New Century returned $54,390.81 to the Dallas Independent School District, making 
the total cost of its program $201,798.69 ($256,189.50 - $54,390.81). 

The 1.2 adjustment takes into account vacant program slots. The cost to the Dallas 
Independent School District would have been much less had not students achievpid so very 
high on InVerijn Performance Objective Tests; e.g., 315 or 334 scored better than 75% 
in reading. 

Thiokol returned $1,728.43 to the District, but their refund was actually $19,892.4 
The Dallas Independent School District did not make the final payment to Thiokol in 
the amount of $18,164.00. 

The education auditor was Educational Testing Service. The primary task of an 
auditor is to **keep everyone honest." The auditor sees that conditions of the contract 
are met by both parties. In the case of this experiment, the auditor certified all 
test and attendaixce results. He monitored testing sessions, attested to the adequacy 
of testing conditions, and certified the items used in interim performance objective 
tests. In the case of vocational subjects, he sought out "expeits" to check the indi- 
vidual progress of students. 

In addition, the auditor attested to the appropriateness and adequacy of the 



-139- 



research design/ He performed curriculum audits to insure that the contractor was not 
"teaching the test." He submitted four thoroughly pointed reports during the course 
of the experiment. The audit contract was in the amount of $27,500.00. 



NOTES FOR A EULOGY OF » PERFORMANCE CONTRACTING' 

By: 

/ JAMES A. MECKLENBURGER 

Critic/researcher, Phi Delta Kappan 

An array o£ things I have wanted to say has been running through my head ever 
since Dick Anderson invited me to write the concluding paper for this conference. 
I thought it might be ii.tportant to say all these things, here, now, because this may 
well be the final forum from which to speak about perfomance contracting. 

But I couldn't write one coherent speech. Five attempts yielded portions of 
five speeches. So, this morning at 4 A.M., I awoke with a panicked urgency to write 
th^t one cohesive masterpiece-of-a-statement which would say everything- there -is -to- 
be -said. 

Like some performance contractors who had partial successes but failed to reach 
their guarantee, I can't deliver that gem- of- a- speech I'd promised myself, but here 
are five partial. speeches that I hope will be of value. 

I, The Last Word on Performance Contracting 

Today's is probably the last word on performance contracting by me. It has 
been a pleasure to be associated with this topic, to have met the people involved and, 
perhaps, as reporter and student, to have helped move the phenomenon along. But we've 
all said about enough. With all respect to Dick Anderson and those sponsoring this 
conference, the small number of people vAio attended is, in itself, a sign that the 
wave of interest in performance contracting as an issue has already passed. Like 
those who rnght have attended but didn't, we have reached the point where we ought 
to stop thinking about the future of performance contracting, to relegate that term 
and some of its implications to a past tense, and to seek more vital issues for our 
attention. 

A second "sign" that performance contracting has peaked as an issue is the 
change of terminology, which Joan Webster told us about, in Grand Rapids. The first 
ERIC year they talked about "performance contracting," the second year about "contract 
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leaming," and this year about * 'individualized instruction." One cannot deny a 
certain amount o£ politics in this shift in termi-nology; nevertheless, that's an 
important shift: the first emphasis was on the device or gimmick of the contract, but 
by the third year emphasis was on what students and teachers ought to be doing in 
schools . 

A third sign is in some of the statements of other speakers in this conference • 
Alex Canja told us that performance contracting is a "process." John Porter, in 
the speech Alex relayed to us, reminded us of the clarification in the Rand Corpora- 
tion study of performance contracting that it is "not a program but a method of 
organizing programs," and George Hall reiterated this. These remarks indicate that 
performance contracting has been reduced from a faddish issue to more nearly its 
proper proportion. We might remember what Charles Blaschke has said about it since he 
introduced the concept in Texarkana three years ago: performance contracting is a 
limited tool. All too often, the people who have written about performance contracting 
have dealt with it as a global topic, not a limited one; with the .passing of a fad we 
should welcome this kind of mistake's being corrected. 

We might learn from other innovations during the 1960 's which passed through a 
faddish stage - team teaching, for exanple, or programmed instruction - which are 
still in use, perhaps moreso than when they were fads, but which no longer stir major 
conflicts where they are used. At one time, like performance contracting, these ideas 
would have become the salvation of public education, according to advocates, or been 
the damnation of it, according to critics. With the passing of these fads, some 
utility was identified in these no- longer- innovative practices. Probably performance 
contracts have some utility and, similarly, will not disappear. 

During this conference we have heard hints of the utility of performance con- 
tracting. It has served in some places as a change agent, if nothing else. Perhaps 
it may become a sales device for new educational practices. It raaji become a ' » 
contracting format tliat will be utilized by teachers in collective negotiations, 
£P^(];ading to the kind of Readier 
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cost-sharing that Charles Blaschke predicted here. 

My dissertation suggests that performance contracting can be likened to drama. 
As with a play, performance contracting is a small thing, but because of the way it's 
honed, it captures our attention and teaches us things about ourselves. Let me read 
a short passage: 

If, as Hamlet tliought, revolutions are in the mind, 
and one can connive with a mere play to "catch the 
conscience of the king," then performance contracts 
have acted in a revolutionary fashion. Tliey offer 
a mirror into which public education has peered 
with chagrin. In this play, contractors did what 
public schools are reluctant to do: made promises 
about sttident learning they would be expected to 
fulfill. Theirs was a pragmatic, impatient, sys- 
tematic approach to schooling; it was bolder, riskier, 
and more political than most educators would have 
found comfortable. 

If one believes that schools should be effective in 
their tasks, then American education is better off 
for the example of performance contracting - however 
imperfectly realized in practice, and whether or 
not it continues - than had it not occurred at all. 
How much has been learned in the mirror, time will 
tell. 

Some evidence of learning through the mirror might be seen in the responses of 
some of performance contracting 's harshest critics, the American Federation of Teachers. 
Bob Bhaerman, Research Director for the AFT, recently told a coi /^^rence in Washington 
that performance contracting has had "some value" in "forcing us - teachers, adminis- 
trators, and businessmen - to take a look at our objectives" as well as at standardized 
testing. At this conference some of you may have noticed Barry Noack, an AFT vice- 
president, involved in sensible discussion with Brian Fitch, Center Manager at Banneker 
School in Gary, Indiana. • 

II. Tlie Definition of "Performance Contract" 

This may be locking the bam after the cow is gone, because this should have 
been settled two or three years ago but has never been. 
Q There is a quirk in our language v\;hich traps us as we talk about "performance 
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Han^ contracting." In English, in our common discourse as we talk^ajnpfig ourselves, argue 



-143- 



and discuss, we frequently use one term (that is, lexical item, word) to mean many 
different thingi^. But we are lax about differentiating meanings and assume eacli 
person knows what another means. 

For example, think of '^marriage/* We know three distinct categories of meaning for 
that term, any of which makes sense in such utterances as "marriage is beautiful." 

"Marriage" is an idea, a concept, capable of being considered abstractly. So 
is "performance contracting." ^Marriage" also denotes a process whose parts we can 
dilineate: get a license, engage a minister, dress in costume, say "I Do." So does 
"performance contracting": RFP, contract, evaluation, payment, turnkey. 

The third stage is experiential. Just as many of us have experienced '^marriage" 
most of us at this conference have lived through a portion of the performance con- 
tract experience. We have seen one, been at one, fought through one, argued over 
one. We look at Texarkana, Gary, or Dade County and say, "I know about performance 
contracting; I've been there." 

Steve Klein made a similar point, in his discussion of the confusions in the OEO 
study, when he compared the question "Does performance contracting work?" to the 
question "Do books work?". "Books" has the same three categories of meaning - concept, 
process, experience. Without differentiation of meanings, one is prone to make con- 
fusing or meaningless utterances. 

OEO did not define what it meant by "performance contracting" when it asked 
*T)oes performance contracting work?". 

Similarly, in this meeting and in dozens of publications, writers and speakers 
make statements about "performance contracting" without specifying whether their 
interest is in the concept, the process, or the experience. This failing, I suggest, . 
is more responsible tlxan anything else for the confusion, anger, misunderstanding and 
disagreement that has greeted this innovation. All too often, people have simply 
been talking to each other about different things in the identical lan.guage. 
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III, The History of Current Performajice Contracting 

This slice of a speech deals with a short seminal period in the history of 
performance contracting which has received relatively little attention. Let me try 
to encapsule that period and what it meajis. 

Remember we've only been at performance contracting for three yearf , and even that 
number is misleading since decision points - should we enter a performance contract - 
have only occurred a couple of times. Yet we^ve already begun to forget why it was, 
in 1970, that people rushed to enter that sequence of processes we've called '*per- 
formance contracting/^ 

In 1970, 50 contracts began; there had been reliable estimates of 150. To 
explain this mushrooming number - since there had been only one contract in 1969 - 
we have to look bade at the short space of time between January and April of 1970 in 
which was created an attitude or flavor. Some have called it a bandwagon or a fad. 
Events caught people in a kind of irrational behavior, a mild mass hysteria; the 
projects in Virginia, the ones in Dallas, the one in Gary, the OEO project and most 
others came out of this environment. 

For six months of the Dorsett project 7 that is, the 1969-1970 contract in 
Texarkana - testing was not at issue. Students entered the learning center and took 
a standardized achievement test. Several hours of instruction later, they took another 
test. Students who achieved at least one year's gain in reading and mathematics - that 
enough for the contractor to receive payment - were returned to regular classes and 
replaced by another student. 

At that time no one asked- the question whether one could do this reliably or 
validly; that is all after the fact. 

In December, Dorsett chose 27 students and administered s standardized test. 
Although the results v^ere said to be for project management purposes only, they were 
published outside of Texarkana because, obviously,, people who were excited about 
performance contracting in Texarkana, including Charles Blarchke, wanted to make it 
known. This is a statement that Blaschke wi^ote at that time: 
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The average grade level increase in reading skills o£ 
all those properly assigned students who were tested with 
alternate forms o£ the same test instrument (for pre- 
and post- testing) was 2.01 grade levels in reading and 
1.09 levels in mathematics. Assuming that 28 hours were 
spent on reading study and 20 hours were spent on mathe- 
matics, this implies one grade level increase per 14 hours 
in reading and per 18 hours in mathematics. 

If you know anything at all about conpensatory education, you can understand why that 

was exciting. And people did get excited. 

In February, 51 students, after 89 total hours of instruction, were achieving 
.99 grade level increases in mathematics, 1.5 in reading. In March, 45 students, 
after a total of 120 hours of instiriction, were achieving 2.2 grade level increases 
in reading and 1.4 in mathematics. 

At that point. Senator George Murphy read the February test scores into the 
Congressional Record . Education USA reported them. Martin J. Filagamo, the project 
director in Texarkana, reported the March test scores in Today *s Education in May, 
which reaches a million readers. Blaschke, Leon Lessinger and others were running 
around the country speaking at any conference they were able. They spread the word: 
there was this thing going on in Texarkana and it was great . They were doing things 
no one had done before, and it was just wonderful. 

Couple the kind of enthusiasm, happiness, elation that we'd finally broken through 
with a kind of ignorance about- standardized testing that we've only begun to break 
through, and you' get a 'Wowl We can do this thingi" 

It isn't very surprising that the OEO people, for exan^ile, were convinced that a 
company could come in and guarantee a year-and-a-half grade gain in one year. Dorsett, 
after all, had done twice that in half-a-year. My suspicion is that had it not been 
for the Dorsett teaching- to-the- test scandal in the summer of 1970, there would have 
been a lot more performance contracts that year (and i£ that scandal had broken a 
week earlier, we might have missed the OEO e^q)eriment, but they announced it before 
the scandal so they were stuck) . 
O What you have to know about the Texarkana scores, however, is that 351 students 

ERIC 

h r i. m..mfT . Ti« t tended the Rapid Learning Centers from October to May, while the number tested in 



Deceinber, Tcbruary and March (27 + SI + 45) was only 123 students, Tliose were the 
scores tliat v/ere reported. 'Iliey were not averages for the student population, Tliosc 
were the students who made it first. But peojile got excited, going what Stan Rlam 
called "stark raving Texarkana'' and tlie bandwagon was off and running, 

Tliere is a larger implication in all this than merely that some of us were either 
hoodwinked or taken for a ride or caught up in something done with integrity. There 
is a problem in education about knowing what is going on. Tliere is no one out there 
who reports authoritatively on what is going on (although Rand is beginning to take on 
some of that function in-cerjain small areas). But there isn't anyone- out there who 
says, "This is so, we believe it, it's valuable, it's reliable. You can believe it." 

In place of that, there are many people who write and each of us relies on a few 
sources of information, we talk to each other through veils of partial information or 
misinformation, and there is a lot of news management, a la Pentagon news management, 
so that we don't really know what is going on and it is easy to start a bandwagon. 

Gary Saretsky, whom Charles Blaschke mentioned earlier, has a way with words. 
Not only did he talk about "Every Kid a Itistler" and the "John Henry Effect", but he 
talked about this being "The Year of the News Release." There was, during 1969, a 
great deal of that kind of managed information. 

Three kinds of information were available during that first year of the Texarkana 
project: 

First, there were the self-congratulatory statements issued by school district 
personnel, by Blaschke, by Lessinger, by Dor sett, by Congressman Pucinski and Senator 
Murphy and others who wished to promote performance contracting, educational technology, 
accountability, or some combination. This was by far the most frequent kind of infor- 
mation vailable. It was general; it was shallous it was biased; it was evangelical. 
But it wat; exciting. 

Second, there were reports of those who visited Texarkana. These tended to be 
school superintendents or their representatives, state and federal officials, education 
"zompanies and their friends, possible clients of Blaschke or Dorsett, and the like. 



Most visitors were friendly toward the project. Jlostile visitors did not go there, since 
no apparent reasons for hostility existed until after the project, when scandal broke. 
Therefore, most people knew almost nothing about the inner workings of the Texarkajia 
project except what its partisans chose to reveal. 

The fact that this is not surprising is, I think, an indictment of the whole 
educational enterprise. We just don't make a practice of collecting and disseminating 
authoritative and truthful data. We don't have to. We treat our data to our advan- 
tage. As a result, people act on misinformation; this goes far beyond performance 
contracting, which serves here as an example. 

IV. Performance Contracting as Essentialism 

Performance contracting, as an historical phenomenon, 1969-1971, has led us down 
a philosophical path which is not implicit in the concept of performance contracting. 

Theodore Brameld once identified four general philosophical approaches to the 
purpose of education: "Perennialism^^ holds that .education's mission is to perpetuate 
the ''perennial'^ heritage of civilized man. '^Essentialism'^ asserts that education should 
provide children the knowledge each must have to function as an adult in his society; 
one ubiquitous example of Essentialism is our contemporary emphasis on 'T^asic skills.'' 
'Trogressivism" emphasizes education's missicn tc develop "the whole child" - a 
philosophical stance rejected by several speakers at this meeting. "Reconstructionism" 
says that education should serve as an instrument for societal changey helping children 
to live in a new and better world. 

Most educators, it seems, accept a blend of these approaches as their own philo- 
sophy; relatively few stress only one of these four viewpoints. But, perhaps by the 
historical accident of its coincidence with compensatory educational programs, perfor- 
mance contracting has appealed to Essentialist elements in each of us, and has attracted 
people who come close to being pure Essentialists in their thinking about education. 

For example, Alex Canja's presentation at this meeting conveyed the Essentialist 
]EP^(^.avor of current performance contracting efforts in Michigan. He told us that what 
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Michigan schools must commit themselves to do is to assure that every cliild learnr> the 
basics that will make him a functioniiig adult in society - specifically to make him 
employable, literate and al^e to calculate; he spoke o£ education's function ab the 
making o£ good citizens for a teclinocratic society. Performance contracting, he told 
this meeting, is a device that will help establish quality control in Michigan's 
schools. (As aji aside, what distressed me in Alex's presentation occurred later, when 
he related to us how he functions in a technocratic society: In his organization - 
which he proudly stated was further ahead than any State Department of Education in the 
nation - everyone must express the identical policy perspective on twenty-nine vital 
educational issues. By juxtaposing his statements, one feels a frightening direction 
toward public schools which serve to train youngsters to function in monolithic 
bureaucratic organizations where everyone is expected to say the same thing.) 

We should stress that nothing in the concept of performance contracting is in- 
herently Essentialist, although there have been few projects to date which have not 
stressed basic skills. One contrary example, which Jack Wilson and I described in the 
September, 1971, Phi Delta Kappan , is a project called "I-Teajn'' in Cherry Creek, 
Colorado. I-Team is a "Progressive" - even "Reconstructionist" - effort of the Cherry 
Creek schools, designed to save potential dropouts by offering them an alternative, 
friendly, SL5)portive educational program. It is a perfoiinance contract; a team of 
teachers agreed to strive to reach nine performance objectives and to receive a bonus 
\:^on the completion of all nine. Some objectives were testable; some were observable; 
some were\judgemental. A team of outside evaluators from universities in Colorado 
agreed to judge the teachers^ success on the nine objectives. The evalioators used 
interviews, personal observation, criterion-referenced tests, final reports by students 
and teachers, and a series of attitude scales, as well as standardized tests. Even- 
tually, the evaluators reported to tiie school board that the teachers, in their judge- 
ment, had succeeded admirably on all nine objectives (in 1970-1971) and the teachers 
received their contracted- for bonus. 

This one exaanple, as well as the teacher- training performance contract in Duval 
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County, the achievement iTKDtivation contract in Dallas, and the vocational training pro- 
gram in Dallas, suggest that performance contracting can be used with any kind o£ in- 
structional objectives, any kind o£ instruction, and many kinds o£ evaluation. 

Whether one chooses to performance contract need not be a product of one's 
philosophy, of one's interest in basic skills or in compensatory education. Rather, 
one should consider a performance contract insofar as he believes it has some advantage 
over other kinds of contracts. 

One exaiT^le of such decision-making can be discerned in Joan Webster's discussion 
at this meeting. She told us that she would prefer, when contracting with a profit- 
seeking company, to use a performance contract rather than a fixed-price contract. 
When you can affect their pocketbook, she said, they are more responsive to your needs. 
That kind of motivational effect has been attributed by many observers to performance 
contracts. It is unrelated to i4iat| the contractor teaches or to whom or by whatever 
method. It relates only to the relative merits of one contracting medium or another. 

When we perceive that performance contracts are somehow inextricably related to 
compensatory education or to basic skills instruction, we may be historically correct, 
but we do the concept of performance contracting a disservice. 

V. Derigibles and Contracts 

Probably this final speech has no great meaning; I'm not sure. But comparing 
derigibles to performance contracting - and perhaps the Hindenberg disaster to the OEO 
experiment - seemed an interesting metaphor to play with, so let's see where it may 
lead* 

Remember the derigibles? Earlier in this century, some people had an exciting 
idea: motorized, long-distance, lighter-than-air vehicles. Not millions of people, 
just a few people. Since no one had flown a whatchamacallit, or built one or seen one, 
these visionaries probably had difficulty even discussing their prized idea, much less 
^ -elling it to others. Imagine trying to excite investors with a flying metal balloon 
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^taith an engine in the back like a submarine. Still, however imperfect their idea. 
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these advocates would have been resolved to overcome the obstacles and create real 
flying submarines. 

This parallels the situation of Giarles Blasclike and a few other people in 1966, 
1967, 1968 and perhaps 1969, and some people even now, in struggling to talk about and 
realize this concept called '^performance contracting," How to describe it: is it 
like business? Skinnerean conditioning? Defense procurement policies? Or is it new 
and unique? What language to talk about it: pedagese? managerial jargon? Use old 
words with new meanings (such as RFP, or turnkey), or new ones (such as learning 
support contractor, or educational accomplishment audit)? 

Since this coterie had some ingenuity, they eventually tried to build tl\eir 
device. They devoted their personal energy, time and commitment to it, regardless 
of how strange it seemed to others (a derigible looks like a lopsided football with 
wings, after all,) • 

If "funny- looking'* is an ignorant criticism of derigibles, it's easy to make and 
hard to refute. Similarly, in 1969-1970, criticisms of performance contracts were 
mostly created out of fear and ignorance. People had no experience with this what- 
chamacallit, but it looked peculiar, sounded lamebrained when described, and called to 
mind possibly hideous consequences: remember how, if Texarkana succeeded, evil 
ravenous corporations would rape the public purse, robotize our children, and make 
public-education-as-we-know-and- love-it disappear? Of course, amidst the negative 
response was some criticism of value: derigibles were dangerous because of inflammable 
gas; performance contracts were suspicious because of their use of standardized tests. 
But these criticisms were, at first, easy to ignore. 

Regardless of hostile criticism, a few whatchamacallits were built. More people 
heard of the idea, some liked it, and a second generation was designed. Soon, -here 
grew a cluster of people, experts, specialists, who really knew about derigibles ~ who 
could fly them, design them, repair them, advertise them, promote them, schedule them, 
Tinance them, etc, 

a I would like to suggest that many of those who attended this^ conference, myself 
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included, are within that coterie of people who have clustered around this derigible, 
performajice contracting, as it grew. 

With initial success came rationalization as we boasted o£ the essential principles 
behind our effort. Lighter- than-air travel: an elementary, fundamental, crisp, lovely 
principle. Payment-for-results : an elementary, fundamental, American- as -apple -pie 
principle, as old as the Greeks. (Mercenaries, we recall, if they died or lost in 
battle, didn't get paid.) Recourse to principle is a pitch for respectability. Some 
unkindly called it * Trucks terism.^' 

New phenomena attract ideas. As this conference has made clear, performance 
contracting has had no surfeit of ideas associated with it. We've heard people say, 
'TVhen you talk about performance contracting, the real issue is . . ." And so, as the 
phenomenon grew, "the real issue" category came to include business and the^profit 
motive, accountability and taxpayer revolt, urban school decay and rising welfare 
roles, contingency management and cost-effectiveness, national politics and dehumani- 
zation of classrooms, and more. As our whatchamacallit became associated with these 
concerns, we who form the innovative coterie have found ourselves pontificating on 
these topics; and who can deny the pleasures of notoriety. One can hear now some 
1930 's radio interviewer earnestly asking the pilot of a derigible what he predicts 
for the future growth of world government. 

What eventually happened is that a prominent whatchamacallit crashed, and ex- 
ploded in front of a news camera. 

Shock. Fear and trembling. What happened? It's awful. It was terrible! 
It was providential. It was only a fluke] Don't worry. We'll try again. We'll do 
it differently. We'll keep going, despite the setback. We'll never see its likes 
again. Will there be funds? Wouldn't touch it with a ten- foot pole. What now? 

I would like to suggest that this conference represents the derigible inventors 
and their followers, meeting a few weeks after the Hindenberg exploded. Minimally, 
most of us are here because we may rtpt have a chance to see each other again. We 
*^ave chosen to talk mostly about old times - very different from the way we used to 
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talk. We laave a new perspective now, since OEO burst our balloon. IVhere we once 
were excited about our whatchamacallit and constantly thought about its future, suddenly 
we have to think about both past and future. Where liave we been? Did we do anything 
that's still important? What have we to salvage? We have personal worries now. 
How shall we make our money next week? Who shall we work for? What shall we write 
about now? How can we maintain our reputation? Whose back might we scratch? We 
are no different, of course, from the derigible pilots, designers, schedulers, 
financiers, etc. 

We evidence a kind of enniu, at this meeting, a kind of lack of commitment and 
direction. Some people have said, "Out of this whatchamacallit experience we have 
learned THIS, %here THIS is how they plan to make their living. Others have said, 
"Give me another chance; I'll design it slightly differently and think up a new name." 
Others have said, "I wasn't really about whatchamacallit anyway - what I really was 
about was whodathunkits." There's been a lot of scrambling going on, and rightfully 
so, for we don't know exactly where we are. 

If there's any hope in this metaphor, other than the promise of nostalgia- to-be 
for the good-old-derigible-days , it is that while derigibles didn't last, lighter-than- 
air travel has lasted. And long-distance air travel has lasted. And a variation of 
the derigible has lasted, in the blimp, although it was never very popular. And now, 
35 years later, there's some enterprising soul who thinks he can use a different gas 
which won't explode, and try derigibles again. Who knows. Maybe, when we're all 
74 or 89, we'll be able to sit crotchety in our rockers and warn cantankerously that 
education has seen this whatchamacallit before. (And we will be accorded the same 
treatment as those contemporary sages who are saying to us: "remember Accountability 
in Victorian England" and "remember Scientific Management," We'll be ignored.) 
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